App Engine Python SDK  v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
Classes | Public Member Functions | Static Public Attributes | List of all members
google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase Class Reference
Inheritance diagram for google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase:
google.appengine.ext.mapreduce.output_writers.OutputWriter google.appengine.ext.mapreduce.json_util.JsonMixin google.appengine.ext.mapreduce.output_writers.BlobstoreOutputWriterBase google.appengine.ext.mapreduce.output_writers.FileOutputWriter google.appengine.ext.mapreduce.output_writers.FileRecordsOutputWriter google.appengine.ext.mapreduce.output_writers.BlobstoreOutputWriter google.appengine.ext.mapreduce.output_writers.BlobstoreRecordsOutputWriter google.appengine.ext.mapreduce.output_writers.KeyValueBlobstoreOutputWriter google.appengine.ext.mapreduce.shuffler._HashingBlobstoreOutputWriter google.appengine.ext.mapreduce.output_writers.BlobstoreOutputWriter google.appengine.ext.mapreduce.output_writers.BlobstoreRecordsOutputWriter google.appengine.ext.mapreduce.output_writers.KeyValueFileOutputWriter google.appengine.ext.mapreduce.output_writers.KeyValueBlobstoreOutputWriter

Classes

class  _State
 

Public Member Functions

def __init__
 
def validate
 
def init_job
 
def finalize_job
 
def from_json
 
def to_json
 
def create
 
def finalize
 
def get_filenames
 
- Public Member Functions inherited from google.appengine.ext.mapreduce.output_writers.OutputWriter
def validate
 
def init_job
 
def finalize_job
 
def from_json
 
def to_json
 
def create
 
def write
 
def finalize
 
def get_filenames
 
- Public Member Functions inherited from google.appengine.ext.mapreduce.json_util.JsonMixin
def to_json_str
 
def from_json_str
 

Static Public Attributes

string OUTPUT_SHARDING_PARAM = "output_sharding"
 
string OUTPUT_SHARDING_NONE = "none"
 
string OUTPUT_SHARDING_INPUT_SHARDS = "input"
 
string OUTPUT_FILESYSTEM_PARAM = "filesystem"
 
string GS_BUCKET_NAME_PARAM = "gs_bucket_name"
 
string GS_ACL_PARAM = "gs_acl"
 

Detailed Description

Base class for all file output writers.

Constructor & Destructor Documentation

def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.__init__ (   self,
  filename,
  request_filename 
)
Init.

Args:
  filename: writable filename from Files API.
  request_filename: in the case of GCS files, we need this to compute
finalized filename. In the case of blobstore, this is useless as
finalized filename can be retrieved from a Files API internal
name mapping.

Member Function Documentation

def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.create (   cls,
  mr_spec,
  shard_number,
  shard_attempt,
  _writer_state = None 
)
Inherit docs.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.finalize (   self,
  ctx,
  shard_state 
)
Finalize writer shard-level state.

Args:
  ctx: an instance of context.Context.
  shard_state: shard state.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.finalize_job (   cls,
  mapreduce_state 
)
See parent class.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.from_json (   cls,
  state 
)
Creates an instance of the OutputWriter for the given json state.

Args:
  state: The OutputWriter state as a json object (dict like).

Returns:
  An instance of the OutputWriter configured using the values of json.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.get_filenames (   cls,
  mapreduce_state 
)
See parent class.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.init_job (   cls,
  mapreduce_state 
)
Initialize job-level writer state.

Args:
  mapreduce_state: an instance of model.MapreduceState describing current
  job.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.to_json (   self)
Returns writer state to serialize in json.

Returns:
  A json-izable version of the OutputWriter state.
def google.appengine.ext.mapreduce.output_writers.FileOutputWriterBase.validate (   cls,
  mapper_spec 
)
Validates mapper specification.

Args:
  mapper_spec: an instance of model.MapperSpec to validate.

Raises:
  BadWriterParamsError: if the specification is invalid for any reason.

The documentation for this class was generated from the following file: