App Engine Python SDK  v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
Public Member Functions | Static Public Attributes | List of all members
google.appengine.ext.mapreduce.input_readers.RecordsReader Class Reference
Inheritance diagram for google.appengine.ext.mapreduce.input_readers.RecordsReader:
google.appengine.ext.mapreduce.input_readers.InputReader google.appengine.ext.mapreduce.json_util.JsonMixin google.appengine.ext.mapreduce.input_readers._ReducerReader google.appengine.ext.mapreduce.shuffler._BatchRecordsReader

Public Member Functions

def __init__
 
def __iter__
 
def from_json
 
def to_json
 
def split_input
 
def validate
 
def __str__
 
- Public Member Functions inherited from google.appengine.ext.mapreduce.input_readers.InputReader
def __iter__
 
def next
 
def from_json
 
def to_json
 
def split_input
 
def validate
 
- Public Member Functions inherited from google.appengine.ext.mapreduce.json_util.JsonMixin
def to_json_str
 
def from_json_str
 

Static Public Attributes

string FILE_PARAM = "file"
 
string FILES_PARAM = "files"
 
- Static Public Attributes inherited from google.appengine.ext.mapreduce.input_readers.InputReader
 expand_parameters = False
 
string NAMESPACE_PARAM = "namespace"
 
string NAMESPACES_PARAM = "namespaces"
 

Detailed Description

Reader to read a list of Files API file in records format.

The number of input shards can be specified by the SHARDS_PARAM
mapper parameter. Input files cannot be split, so there will be at most
one shard per file. Also the number of shards will not be reduced based on
the number of input files, so shards in always equals shards out.

Constructor & Destructor Documentation

def google.appengine.ext.mapreduce.input_readers.RecordsReader.__init__ (   self,
  filenames,
  position 
)
Constructor.

Args:
  filenames: list of filenames.
  position: file position to start reading from as int.

Member Function Documentation

def google.appengine.ext.mapreduce.input_readers.RecordsReader.__iter__ (   self)
Iterate over records in file.

Yields:
  Records as strings.
def google.appengine.ext.mapreduce.input_readers.RecordsReader.from_json (   cls,
  json 
)
Creates an instance of the InputReader for the given input shard state.

Args:
  json: The InputReader state as a dict-like object.

Returns:
  An instance of the InputReader configured using the values of json.
def google.appengine.ext.mapreduce.input_readers.RecordsReader.split_input (   cls,
  mapper_spec 
)
Returns a list of input readers for the input spec.

Args:
  mapper_spec: The MapperSpec for this InputReader.

Returns:
  A list of InputReaders.
def google.appengine.ext.mapreduce.input_readers.RecordsReader.to_json (   self)
Returns an input shard state for the remaining inputs.

Returns:
  A json-izable version of the remaining InputReader.
def google.appengine.ext.mapreduce.input_readers.RecordsReader.validate (   cls,
  mapper_spec 
)
Validates mapper spec and all mapper parameters.

Args:
  mapper_spec: The MapperSpec for this InputReader.

Raises:
  BadReaderParamsError: required parameters are missing or invalid.

The documentation for this class was generated from the following file: