google.cloud.bigquery.job.LoadJob#

Methods

add_done_callback(fn)

Add a callback to be executed when the operation is complete.

cancel([client])

API call: cancel job via a POST request

cancelled()

Check if the job has been cancelled.

done([retry])

Refresh the job and checks if it is complete.

exception([timeout])

Get the exception from the operation, blocking if necessary.

exists([client, retry])

API call: test for the existence of the job via a GET request

from_api_repr(resource, client)

Factory: construct a job given its API representation

reload([client, retry])

API call: refresh job properties via a GET request.

result([timeout, retry])

Start the job and wait for it to complete and get the result.

running()

True if the operation is currently running.

set_exception(exception)

Set the Future’s exception.

set_result(result)

Set the Future’s result.

to_api_repr()

Generate a resource for _begin().

Attributes

allow_jagged_rows

See google.cloud.bigquery.job.LoadJobConfig.allow_jagged_rows.

allow_quoted_newlines

See google.cloud.bigquery.job.LoadJobConfig.allow_quoted_newlines.

autodetect

See google.cloud.bigquery.job.LoadJobConfig.autodetect.

clustering_fields

See google.cloud.bigquery.job.LoadJobConfig.clustering_fields.

create_disposition

See google.cloud.bigquery.job.LoadJobConfig.create_disposition.

created

Datetime at which the job was created.

destination

table where loaded rows are written

destination_encryption_configuration

Custom encryption configuration for the destination table.

destination_table_description

Union[str, None] name given to destination table.

destination_table_friendly_name

Union[str, None] name given to destination table.

encoding

See google.cloud.bigquery.job.LoadJobConfig.encoding.

ended

Datetime at which the job finished.

error_result

Error information about the job as a whole.

errors

Information about individual errors generated by the job.

etag

ETag for the job resource.

field_delimiter

See google.cloud.bigquery.job.LoadJobConfig.field_delimiter.

ignore_unknown_values

See google.cloud.bigquery.job.LoadJobConfig.ignore_unknown_values.

input_file_bytes

Count of bytes loaded from source files.

input_files

Count of source files.

job_id

ID of the job.

job_type

Type of job

labels

Labels for the job.

location

Location where the job runs.

max_bad_records

See google.cloud.bigquery.job.LoadJobConfig.max_bad_records.

null_marker

See google.cloud.bigquery.job.LoadJobConfig.null_marker.

output_bytes

Count of bytes saved to destination table.

output_rows

Count of rows saved to destination table.

path

URL path for the job’s APIs.

project

Project bound to the job.

quote_character

See google.cloud.bigquery.job.LoadJobConfig.quote_character.

schema

See google.cloud.bigquery.job.LoadJobConfig.schema.

schema_update_options

See google.cloud.bigquery.job.LoadJobConfig.schema_update_options.

self_link

URL for the job resource.

skip_leading_rows

See google.cloud.bigquery.job.LoadJobConfig.skip_leading_rows.

source_format

See google.cloud.bigquery.job.LoadJobConfig.source_format.

started

Datetime at which the job was started.

state

Status of the job.

time_partitioning

See google.cloud.bigquery.job.LoadJobConfig.time_partitioning.

use_avro_logical_types

See google.cloud.bigquery.job.LoadJobConfig.use_avro_logical_types.

user_email

E-mail address of user who submitted the job.

write_disposition

See google.cloud.bigquery.job.LoadJobConfig.write_disposition.


class google.cloud.bigquery.job.LoadJob(job_id, source_uris, destination, client, job_config=None)[source]#

Bases: google.cloud.bigquery.job._AsyncJob

Asynchronous job for loading data into a table.

Can load from Google Cloud Storage URIs or from a file.

Parameters
add_done_callback(fn)#

Add a callback to be executed when the operation is complete.

If the operation is not already complete, this will start a helper thread to poll for the status of the operation in the background.

Parameters

fn (Callable[Future]) – The callback to execute when the operation is complete.

property allow_jagged_rows#

See google.cloud.bigquery.job.LoadJobConfig.allow_jagged_rows.

property allow_quoted_newlines#

See google.cloud.bigquery.job.LoadJobConfig.allow_quoted_newlines.

property autodetect#

See google.cloud.bigquery.job.LoadJobConfig.autodetect.

cancel(client=None)#

API call: cancel job via a POST request

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/cancel

Parameters

client (Client or NoneType) – the client to use. If not passed, falls back to the client stored on the current dataset.

Return type

bool

Returns

Boolean indicating that the cancel request was sent.

cancelled()#

Check if the job has been cancelled.

This always returns False. It’s not possible to check if a job was cancelled in the API. This method is here to satisfy the interface for google.api_core.future.Future.

Return type

bool

Returns

False

property clustering_fields#

See google.cloud.bigquery.job.LoadJobConfig.clustering_fields.

property create_disposition#

See google.cloud.bigquery.job.LoadJobConfig.create_disposition.

property created#

Datetime at which the job was created.

Return type

datetime.datetime, or NoneType

Returns

the creation time (None until set from the server).

property destination#

table where loaded rows are written

See: https://g.co/cloud/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationTable

Type

google.cloud.bigquery.table.TableReference

property destination_encryption_configuration#

Custom encryption configuration for the destination table.

Custom encryption configuration (e.g., Cloud KMS keys) or None if using default encryption.

See google.cloud.bigquery.job.LoadJobConfig.destination_encryption_configuration.

Type

google.cloud.bigquery.table.EncryptionConfiguration

property destination_table_description#

Union[str, None] name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationTableProperties.description

property destination_table_friendly_name#

Union[str, None] name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationTableProperties.friendlyName

done(retry=<google.api_core.retry.Retry object>)#

Refresh the job and checks if it is complete.

Parameters

retry (google.api_core.retry.Retry) – (Optional) How to retry the RPC.

Return type

bool

Returns

True if the job is complete, False otherwise.

property encoding#

See google.cloud.bigquery.job.LoadJobConfig.encoding.

property ended#

Datetime at which the job finished.

Return type

datetime.datetime, or NoneType

Returns

the end time (None until set from the server).

property error_result#

Error information about the job as a whole.

Return type

mapping, or NoneType

Returns

the error information (None until set from the server).

property errors#

Information about individual errors generated by the job.

Return type

list of mappings, or NoneType

Returns

the error information (None until set from the server).

property etag#

ETag for the job resource.

Return type

str, or NoneType

Returns

the ETag (None until set from the server).

exception(timeout=None)#

Get the exception from the operation, blocking if necessary.

Parameters

timeout (int) – How long to wait for the operation to complete. If None, wait indefinitely.

Returns

The operation’s

error.

Return type

Optional[google.api_core.GoogleAPICallError]

exists(client=None, retry=<google.api_core.retry.Retry object>)#

API call: test for the existence of the job via a GET request

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/get

Parameters
  • client (Client or NoneType) – the client to use. If not passed, falls back to the client stored on the current dataset.

  • retry (google.api_core.retry.Retry) – (Optional) How to retry the RPC.

Return type

bool

Returns

Boolean indicating existence of the job.

property field_delimiter#

See google.cloud.bigquery.job.LoadJobConfig.field_delimiter.

classmethod from_api_repr(resource, client)[source]#

Factory: construct a job given its API representation

Parameters
Return type

google.cloud.bigquery.job.LoadJob

Returns

Job parsed from resource.

property ignore_unknown_values#

See google.cloud.bigquery.job.LoadJobConfig.ignore_unknown_values.

property input_file_bytes#

Count of bytes loaded from source files.

Return type

int, or NoneType

Returns

the count (None until set from the server).

Raises

ValueError for invalid value types.

property input_files#

Count of source files.

Return type

int, or NoneType

Returns

the count (None until set from the server).

property job_id#

ID of the job.

Type

str

property job_type#

Type of job

Return type

str

Returns

one of ‘load’, ‘copy’, ‘extract’, ‘query’

property labels#

Labels for the job.

Type

Dict[str, str]

property location#

Location where the job runs.

Type

str

property max_bad_records#

See google.cloud.bigquery.job.LoadJobConfig.max_bad_records.

property null_marker#

See google.cloud.bigquery.job.LoadJobConfig.null_marker.

property output_bytes#

Count of bytes saved to destination table.

Return type

int, or NoneType

Returns

the count (None until set from the server).

property output_rows#

Count of rows saved to destination table.

Return type

int, or NoneType

Returns

the count (None until set from the server).

property path#

URL path for the job’s APIs.

Return type

str

Returns

the path based on project and job ID.

property project#

Project bound to the job.

Return type

str

Returns

the project (derived from the client).

property quote_character#

See google.cloud.bigquery.job.LoadJobConfig.quote_character.

reload(client=None, retry=<google.api_core.retry.Retry object>)#

API call: refresh job properties via a GET request.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/get

Parameters
  • client (Client or NoneType) – the client to use. If not passed, falls back to the client stored on the current dataset.

  • retry (google.api_core.retry.Retry) – (Optional) How to retry the RPC.

result(timeout=None, retry=<google.api_core.retry.Retry object>)#

Start the job and wait for it to complete and get the result.

Parameters
Return type

_AsyncJob

Returns

This instance.

Raises

GoogleCloudError if the job failed or concurrent.futures.TimeoutError if the job did not complete in the given timeout.

running()#

True if the operation is currently running.

property schema#

See google.cloud.bigquery.job.LoadJobConfig.schema.

property schema_update_options#

See google.cloud.bigquery.job.LoadJobConfig.schema_update_options.

URL for the job resource.

Return type

str, or NoneType

Returns

the URL (None until set from the server).

set_exception(exception)#

Set the Future’s exception.

set_result(result)#

Set the Future’s result.

property skip_leading_rows#

See google.cloud.bigquery.job.LoadJobConfig.skip_leading_rows.

property source_format#

See google.cloud.bigquery.job.LoadJobConfig.source_format.

property started#

Datetime at which the job was started.

Return type

datetime.datetime, or NoneType

Returns

the start time (None until set from the server).

property state#

Status of the job.

Return type

str, or NoneType

Returns

the state (None until set from the server).

property time_partitioning#

See google.cloud.bigquery.job.LoadJobConfig.time_partitioning.

to_api_repr()[source]#

Generate a resource for _begin().

property use_avro_logical_types#

See google.cloud.bigquery.job.LoadJobConfig.use_avro_logical_types.

property user_email#

E-mail address of user who submitted the job.

Return type

str, or NoneType

Returns

the URL (None until set from the server).

property write_disposition#

See google.cloud.bigquery.job.LoadJobConfig.write_disposition.