![]() |
App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
|
Public Member Functions | |
def | __init__ |
def | validate |
def | split_input |
def | next |
def | to_json |
def | from_json |
def | __str__ |
![]() | |
def | __iter__ |
def | next |
def | from_json |
def | to_json |
def | split_input |
def | validate |
![]() | |
def | to_json_str |
def | from_json_str |
Static Public Attributes | |
string | BLOB_KEYS_PARAM = "blob_keys" |
string | BLOB_KEY_PARAM = "blob_key" |
string | START_FILE_INDEX_PARAM = "start_file_index" |
string | END_FILE_INDEX_PARAM = "end_file_index" |
string | OFFSET_PARAM = "offset" |
![]() | |
expand_parameters = False | |
string | NAMESPACE_PARAM = "namespace" |
string | NAMESPACES_PARAM = "namespaces" |
Input reader for newline delimited files in zip archives from Blobstore. This has the same external interface as the BlobstoreLineInputReader, in that it takes a list of blobs as its input and yields lines to the reader. However the blobs themselves are expected to be zip archives of line delimited files instead of the files themselves. This is useful as many line delimited files gain greatly from compression.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.__init__ | ( | self, | |
blob_key, | |||
start_file_index, | |||
end_file_index, | |||
offset, | |||
_reader = blobstore.BlobReader |
|||
) |
Initializes this instance with the given blob key and file range. This BlobstoreZipLineInputReader will read from the file with index start_file_index up to but not including the file with index end_file_index. It will return lines starting at offset within file[start_file_index] Args: blob_key: the BlobKey that this input reader is processing. start_file_index: the index of the first file to read within the zip. end_file_index: the index of the first file that will not be read. offset: the byte offset within blob_key.zip[start_file_index] to start reading. The reader will continue to the end of the file. _reader: a callable that returns a file-like object for reading blobs. Used for dependency injection.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.__str__ | ( | self | ) |
Returns the string representation of this reader. Returns: string blobkey:[start file num, end file num]:current offset.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.from_json | ( | cls, | |
json, | |||
_reader = blobstore.BlobReader |
|||
) |
Creates an instance of the InputReader for the given input shard state. Args: json: The InputReader state as a dict-like object. _reader: For dependency injection. Returns: An instance of the InputReader configured using the values of json.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.next | ( | self | ) |
Returns the next line from this input reader as (lineinfo, line) tuple. Returns: The next input from this input reader, in the form of a 2-tuple. The first element of the tuple describes the source, it is itself a tuple (blobkey, filenumber, byteoffset). The second element of the tuple is the line found at that offset.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.split_input | ( | cls, | |
mapper_spec, | |||
_reader = blobstore.BlobReader |
|||
) |
Returns a list of input readers for the input spec. Args: mapper_spec: The MapperSpec for this InputReader. Must contain 'blob_keys' parameter with one or more blob keys. _reader: a callable that returns a file-like object for reading blobs. Used for dependency injection. Returns: A list of InputReaders spanning the subfiles within the blobs. There will be at least one reader per blob, but it will otherwise attempt to keep the expanded size even.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.to_json | ( | self | ) |
Returns an input shard state for the remaining inputs. Returns: A json-izable version of the remaining InputReader.
def google.appengine.ext.mapreduce.input_readers.BlobstoreZipLineInputReader.validate | ( | cls, | |
mapper_spec | |||
) |
Validates mapper spec and all mapper parameters. Args: mapper_spec: The MapperSpec for this InputReader. Raises: BadReaderParamsError: required parameters are missing or invalid.