![]() |
App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
|
Public Member Functions | |
def | __init__ |
def | UseProgressData |
def | StoreKeys |
def | UpdateState |
def | DeleteKey |
def | GetProgressStatusGenerator |
![]() | |
def | __init__ |
def | ThreadComplete |
Public Attributes | |
prior_key_end | |
py_type | |
![]() | |
db_filename | |
primary_conn | |
primary_thread | |
secondary_conn | |
secondary_thread | |
operation_count | |
commit_periodicity | |
existing_table | |
insert_cursor | |
update_cursor | |
Additional Inherited Members | |
![]() | |
string | SIGNATURE_TABLE_NAME = 'bulkloader_database_signature' |
Persistently record all progress information during an upload. This class wraps a very simple SQLite database which records each of the relevant details from a chunk of work. If the loader is resumed, then data is replayed out of the database.
def google.appengine.tools.bulkloader._ProgressDatabase.__init__ | ( | self, | |
db_filename, | |||
sql_type, | |||
py_type, | |||
signature, | |||
commit_periodicity = 100 |
|||
) |
Initialize the ProgressDatabase instance. Args: db_filename: The name of the SQLite database to use. sql_type: A string of the SQL type to use for entity keys. py_type: The python type of entity keys. signature: A string identifying the important invocation options, used to make sure we are not using an old database. commit_periodicity: How many operations to perform between commits.
def google.appengine.tools.bulkloader._ProgressDatabase.DeleteKey | ( | self, | |
progress_key | |||
) |
Delete the entities with the given key from the result database.
def google.appengine.tools.bulkloader._ProgressDatabase.GetProgressStatusGenerator | ( | self | ) |
Get a generator which yields progress information. The returned generator will yield a series of 5-tuples that specify progress information about a prior run of the uploader. The 5-tuples have the following values: progress_key: The unique key to later update this record with new progress information. state: The last state saved for this progress record. kind: The datastore kind of the items for uploading. key_start: The starting key of the items for uploading (inclusive). key_end: The ending key of the items for uploading (inclusive). After all incompletely-transferred records are provided, then one more 5-tuple will be generated: None DATA_CONSUMED_TO_HERE: A unique string value indicating this record is being provided. None None key_end: An integer value specifying the last data source key that was handled by the previous run of the uploader. The caller should begin uploading records which occur after key_end. Yields: Five-tuples of (progress_key, state, kind, key_start, key_end)
def google.appengine.tools.bulkloader._ProgressDatabase.StoreKeys | ( | self, | |
kind, | |||
key_start, | |||
key_end | |||
) |
Record a new progress record, returning a key for later updates. The specified progress information will be persisted into the database. A unique key will be returned that identifies this progress state. The key is later used to (quickly) update this record. For the progress resumption to proceed properly, calls to StoreKeys MUST specify monotonically increasing key ranges. This will result in a database whereby the ID, KEY_START, and KEY_END rows are all increasing (rather than having ranges out of order). NOTE: the above precondition is NOT tested by this method (since it would imply an additional table read or two on each invocation). Args: kind: The kind for the WorkItem key_start: The starting key of the WorkItem (inclusive) key_end: The end key of the WorkItem (inclusive) Returns: A string to later be used as a unique key to update this state.
def google.appengine.tools.bulkloader._ProgressDatabase.UpdateState | ( | self, | |
key, | |||
new_state | |||
) |
Update a specified progress record with new information. Args: key: The key for this progress record, returned from StoreKeys new_state: The new state to associate with this progress record.
def google.appengine.tools.bulkloader._ProgressDatabase.UseProgressData | ( | self | ) |
Returns True if the database has progress information. Note there are two basic cases for progress information: 1) All saved records indicate a successful upload. In this case, we need to skip everything transmitted so far and then send the rest. 2) Some records for incomplete transfer are present. These need to be sent again, and then we resume sending after all the successful data. Returns: True: if the database has progress information. Raises: ResumeError: if there is an error retrieving rows from the database.