![]() |
App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
|
Public Member Functions | |
def | __init__ |
def | peek |
def | next |
def | consume |
def | consume_if |
def | remainder |
Public Attributes | |
index | |
Static Public Attributes | |
string | SPECIAL_CHARS = '[]()=,' |
string | ESCAPE_CHAR = '\\' |
Tokenizes a user supplied format string. A token is either a special character or a group of characters between two special characters or the beginning or the end of format string. Escape character can be used to escape special characters and itself.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.__init__ | ( | self, | |
format_string | |||
) |
Initialize. Args: format_string: user supplied format string for MapReduce InputReader.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.consume | ( | self, | |
expected_token | |||
) |
Consumes the next token which must match expectation. Args: expected_token: the expected value of the next token. Raises: ValueError: raised when the next token doesn't match expected_token.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.consume_if | ( | self, | |
token | |||
) |
Consumes the next token when it matches expectation. Args: token: the expected next token. Returns: True when next token matches the argument and is consumed. False otherwise.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.next | ( | self, | |
advance = True |
|||
) |
Returns the next token with surrounding white spaces stripped. Args: advance: boolean. True if underlying buffer should be advanced. Returns: the next token with surrounding whitespaces stripped.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.peek | ( | self | ) |
Returns the next token with surrounding white spaces stripped. This method does not advance underlying buffer. Returns: the next token with surrounding whitespaces stripped.
def google.appengine.ext.mapreduce.file_format_parser._Tokenizer.remainder | ( | self | ) |
Returns the number of bytes left to be processed.