![]() |
App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
|
Public Member Functions | |
def | __init__ |
def | validate |
def | init_job |
def | finalize_job |
def | from_json |
def | to_json |
def | create |
def | get_filenames |
def | finalize |
def | write |
![]() | |
def | __init__ |
def | validate |
def | init_job |
def | finalize_job |
def | from_json |
def | to_json |
def | create |
def | finalize |
def | get_filenames |
![]() | |
def | validate |
def | init_job |
def | finalize_job |
def | from_json |
def | to_json |
def | create |
def | write |
def | finalize |
def | get_filenames |
![]() | |
def | to_json_str |
def | from_json_str |
Additional Inherited Members | |
![]() | |
string | OUTPUT_SHARDING_PARAM = "output_sharding" |
string | OUTPUT_SHARDING_NONE = "none" |
string | OUTPUT_SHARDING_INPUT_SHARDS = "input" |
string | OUTPUT_FILESYSTEM_PARAM = "filesystem" |
string | GS_BUCKET_NAME_PARAM = "gs_bucket_name" |
string | GS_ACL_PARAM = "gs_acl" |
An OutputWriter which outputs data into blobstore in key-value format. The output is tailored towards shuffler needs. It shards key/values using key hash modulo number of output files.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.__init__ | ( | self, | |
filenames | |||
) |
Constructor. Args: filenames: list of filenames that this writer outputs to.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.create | ( | cls, | |
mr_spec, | |||
shard_number, | |||
shard_attempt, | |||
_writer_state = None |
|||
) |
Inherit docs.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.finalize_job | ( | cls, | |
mapreduce_state | |||
) |
Finalize job-level writer state. Args: mapreduce_state: an instance of model.MapreduceState describing current job. State can be modified during finalization.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.from_json | ( | cls, | |
json | |||
) |
Creates an instance of the OutputWriter for the given json state. Args: json: The OutputWriter state as a dict-like object. Returns: An instance of the OutputWriter configured using the values of json.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.get_filenames | ( | cls, | |
mapreduce_state | |||
) |
See parent class.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.init_job | ( | cls, | |
mapreduce_state | |||
) |
Initialize job-level writer state. Args: mapreduce_state: an instance of model.MapreduceState describing current job. State can be modified during initialization.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.to_json | ( | self | ) |
Returns writer state to serialize in json. Returns: A json-izable version of the OutputWriter state.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.validate | ( | cls, | |
mapper_spec | |||
) |
Validates mapper specification. Args: mapper_spec: an instance of model.MapperSpec to validate.
def google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter.write | ( | self, | |
data | |||
) |
Write data. Args: data: actual data yielded from handler. Type is writer-specific.