google.cloud.bigquery.job.LoadJobConfig#

Methods

from_api_repr(resource)

Factory: construct a job configuration given its API representation

to_api_repr()

Build an API representation of the job config.

Attributes

allow_jagged_rows

Allow missing trailing optional columns (CSV only).

allow_quoted_newlines

Allow quoted data containing newline characters (CSV only).

autodetect

Automatically infer the schema from a sample of the data.

clustering_fields

Fields defining clustering for the table

create_disposition

Specifies behavior for creating tables.

destination_encryption_configuration

Custom encryption configuration for the destination table.

destination_table_description

Union[str, None] name given to destination table.

destination_table_friendly_name

Union[str, None] name given to destination table.

encoding

The character encoding of the data.

field_delimiter

The separator for fields in a CSV file.

ignore_unknown_values

Ignore extra values not represented in the table schema.

labels

Labels for the job.

max_bad_records

Number of invalid rows to ignore.

null_marker

Represents a null value (CSV only).

quote_character

Character used to quote data sections (CSV only).

schema

Schema of the destination table.

schema_update_options

Specifies updates to the destination table schema to allow as a side effect of the load job.

skip_leading_rows

Number of rows to skip when reading data (CSV only).

source_format

File format of the data.

time_partitioning

Specifies time-based partitioning for the destination table.

use_avro_logical_types

For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types(e.g.

write_disposition

Action that occurs if the destination table already exists.


class google.cloud.bigquery.job.LoadJobConfig(**kwargs)[source]#

Bases: google.cloud.bigquery.job._JobConfig

Configuration options for load jobs.

All properties in this class are optional. Values which are None -> server defaults. Set properties on the constructed configuration by using the property name as the name of a keyword argument.

property allow_jagged_rows#

Allow missing trailing optional columns (CSV only).

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.allowJaggedRows

Type

bool

property allow_quoted_newlines#

Allow quoted data containing newline characters (CSV only).

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.allowQuotedNewlines

Type

bool

property autodetect#

Automatically infer the schema from a sample of the data.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.autodetect

Type

bool

property clustering_fields#

Fields defining clustering for the table

(Defaults to None).

Clustering fields are immutable after table creation.

Note

As of 2018-06-29, clustering fields cannot be set on a table which does not also have time partioning defined.

Type

Union[List[str], None]

property create_disposition#

Specifies behavior for creating tables.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.createDisposition

Type

google.cloud.bigquery.job.CreateDisposition

property destination_encryption_configuration#

Custom encryption configuration for the destination table.

Custom encryption configuration (e.g., Cloud KMS keys) or None if using default encryption.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationEncryptionConfiguration

Type

google.cloud.bigquery.table.EncryptionConfiguration

property destination_table_description#

Union[str, None] name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationTableProperties.description

property destination_table_friendly_name#

Union[str, None] name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.destinationTableProperties.friendlyName

property encoding#

The character encoding of the data.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.encoding

Type

google.cloud.bigquery.job.Encoding

property field_delimiter#

The separator for fields in a CSV file.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.fieldDelimiter

Type

str

classmethod from_api_repr(resource)#

Factory: construct a job configuration given its API representation

Parameters

resource (dict) – An extract job configuration in the same representation as is returned from the API.

Return type

google.cloud.bigquery.job._JobConfig

Returns

Configuration parsed from resource.

property ignore_unknown_values#

Ignore extra values not represented in the table schema.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.ignoreUnknownValues

Type

bool

property labels#

Labels for the job.

This method always returns a dict. To change a job’s labels, modify the dict, then call Client.update_job. To delete a label, set its value to None before updating.

Raises

ValueError – If value type is invalid.

Type

Dict[str, str]

property max_bad_records#

Number of invalid rows to ignore.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.maxBadRecords

Type

int

property null_marker#

Represents a null value (CSV only).

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.nullMarker

Type

str

property quote_character#

Character used to quote data sections (CSV only).

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.quote

Type

str

property schema#

Schema of the destination table.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.schema

Type

List[google.cloud.bigquery.schema.SchemaField]

property schema_update_options#

Specifies updates to the destination table schema to allow as a side effect of the load job.

Type

List[google.cloud.bigquery.job.SchemaUpdateOption]

property skip_leading_rows#

Number of rows to skip when reading data (CSV only).

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.skipLeadingRows

Type

int

property source_format#

File format of the data.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.sourceFormat

Type

google.cloud.bigquery.job.SourceFormat

property time_partitioning#

Specifies time-based partitioning for the destination table.

Type

google.cloud.bigquery.table.TimePartitioning

to_api_repr()#

Build an API representation of the job config.

Return type

dict

Returns

A dictionary in the format used by the BigQuery API.

property use_avro_logical_types#

For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types(e.g. TIMESTAMP) rather than raw types (e.g. INTEGER).

Type

bool

property write_disposition#

Action that occurs if the destination table already exists.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.writeDisposition

Type

google.cloud.bigquery.job.WriteDisposition