App Engine Python SDK  v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
Public Member Functions | Static Public Attributes | List of all members
google.appengine.ext.mapreduce.input_readers.InputReader Class Reference
Inheritance diagram for google.appengine.ext.mapreduce.input_readers.InputReader:
google.appengine.ext.mapreduce.json_util.JsonMixin google.appengine.ext.mapreduce.input_readers._GoogleCloudStorageInputReader google.appengine.ext.mapreduce.input_readers._OldAbstractDatastoreInputReader google.appengine.ext.mapreduce.input_readers.AbstractDatastoreInputReader google.appengine.ext.mapreduce.input_readers.BlobstoreLineInputReader google.appengine.ext.mapreduce.input_readers.BlobstoreZipInputReader google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader google.appengine.ext.mapreduce.input_readers.FileInputReader google.appengine.ext.mapreduce.input_readers.LogInputReader google.appengine.ext.mapreduce.input_readers.NamespaceInputReader google.appengine.ext.mapreduce.input_readers.RandomStringInputReader google.appengine.ext.mapreduce.input_readers.RecordsReader google.appengine.ext.mapreduce.shuffler._MergingReader

Public Member Functions

def __iter__
 
def next
 
def from_json
 
def to_json
 
def split_input
 
def validate
 
- Public Member Functions inherited from google.appengine.ext.mapreduce.json_util.JsonMixin
def to_json_str
 
def from_json_str
 

Static Public Attributes

 expand_parameters = False
 
string NAMESPACE_PARAM = "namespace"
 
string NAMESPACES_PARAM = "namespaces"
 

Detailed Description

Abstract base class for input readers.

InputReaders have the following properties:
 * They are created by using the split_input method to generate a set of
   InputReaders from a MapperSpec.
 * They generate inputs to the mapper via the iterator interface.
 * After creation, they can be serialized and resumed using the JsonMixin
   interface.
 * They are cast to string for a user-readable description; it may be
   valuable to implement __str__.

Member Function Documentation

def google.appengine.ext.mapreduce.input_readers.InputReader.from_json (   cls,
  input_shard_state 
)
Creates an instance of the InputReader for the given input shard state.

Args:
  input_shard_state: The InputReader state as a dict-like object.

Returns:
  An instance of the InputReader configured using the values of json.
def google.appengine.ext.mapreduce.input_readers.InputReader.next (   self)
Returns the next input from this input reader as a key, value pair.

Returns:
  The next input from this input reader.
def google.appengine.ext.mapreduce.input_readers.InputReader.split_input (   cls,
  mapper_spec 
)
Returns a list of input readers.

This method creates a list of input readers, each for one shard.
It attempts to split inputs among readers evenly.

Args:
  mapper_spec: model.MapperSpec specifies the inputs and additional
parameters to define the behavior of input readers.

Returns:
  A list of InputReaders. None or [] when no input data can be found.
def google.appengine.ext.mapreduce.input_readers.InputReader.to_json (   self)
Returns an input shard state for the remaining inputs.

Returns:
  A json-izable version of the remaining InputReader.
def google.appengine.ext.mapreduce.input_readers.InputReader.validate (   cls,
  mapper_spec 
)
Validates mapper spec and all mapper parameters.

Input reader parameters are expected to be passed as "input_reader"
subdictionary in mapper_spec.params.

Pre 1.6.4 API mixes input reader parameters with all other parameters. Thus
to be compatible, input reader check mapper_spec.params as well and
issue a warning if "input_reader" subdicationary is not present.

Args:
  mapper_spec: The MapperSpec for this InputReader.

Raises:
  BadReaderParamsError: required parameters are missing or invalid.

The documentation for this class was generated from the following file: