airflow.contrib.operators.bigquery_to_gcs
¶
This module contains a Google BigQuery to GCS operator.
Module Contents¶
-
class
airflow.contrib.operators.bigquery_to_gcs.
BigQueryToCloudStorageOperator
(source_project_dataset_table, destination_cloud_storage_uris, compression='NONE', export_format='CSV', field_delimiter=', ', print_header=True, gcp_conn_id='google_cloud_default', bigquery_conn_id=None, delegate_to=None, labels=None, *args, **kwargs)[source]¶ Bases:
airflow.models.BaseOperator
Transfers a BigQuery table to a Google Cloud Storage bucket.
See also
For more details about these parameters: https://cloud.google.com/bigquery/docs/reference/v2/jobs
- Parameters
source_project_dataset_table (str) – The dotted
(<project>.|<project>:)<dataset>.<table>
BigQuery table to use as the source data. If<project>
is not included, project will be the project defined in the connection json. (templated)destination_cloud_storage_uris (list) – The destination Google Cloud Storage URI (e.g. gs://some-bucket/some-file.txt). (templated) Follows convention defined here: https://cloud.google.com/bigquery/exporting-data-from-bigquery#exportingmultiple
compression (str) – Type of compression to use.
export_format (str) – File format to export.
field_delimiter (str) – The delimiter to use when extracting to a CSV.
print_header (bool) – Whether to print a header for a CSV file extract.
gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.
bigquery_conn_id (str) – (Deprecated) The connection ID used to connect to Google Cloud Platform. This parameter has been deprecated. You should pass the gcp_conn_id parameter instead.
delegate_to (str) – The account to impersonate, if any. For this to work, the service account making the request must have domain-wide delegation enabled.
labels (dict) – a dictionary containing labels for the job/query, passed to BigQuery