airflow.gcp.operators.tasks

This module contains various GCP Cloud Tasks operators which allow you to perform basic operations using Cloud Tasks queues/tasks.

Module Contents

class airflow.gcp.operators.tasks.CloudTasksQueueCreateOperator(location, task_queue, project_id=None, queue_name=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Creates a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be created.

  • task_queue (dict or class google.cloud.tasks_v2.types.Queue) – The task queue to create. Queue’s name cannot be the same as an existing queue. If a dict is provided, it must be of the same form as the protobuf message Queue.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • queue_name (str) – (Optional) The queue’s name. If provided, it will be used to construct the full queue path.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Queue

template_fields = ['task_queue', 'project_id', 'location', 'queue_name', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueueUpdateOperator(task_queue, project_id=None, location=None, queue_name=None, update_mask=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Updates a queue in Cloud Tasks.

Parameters
  • task_queue (dict or class google.cloud.tasks_v2.types.Queue) – The task queue to update. This method creates the queue if it does not exist and updates the queue if it does exist. The queue’s name must be specified.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • location (str) – (Optional) The location name in which the queue will be updated. If provided, it will be used to construct the full queue path.

  • queue_name (str) – (Optional) The queue’s name. If provided, it will be used to construct the full queue path.

  • update_mask (dict or class google.cloud.tasks_v2.types.FieldMask) – A mast used to specify which fields of the queue are being updated. If empty, then all fields will be updated. If a dict is provided, it must be of the same form as the protobuf message.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Queue

template_fields = ['task_queue', 'project_id', 'location', 'queue_name', 'update_mask', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueueGetOperator(location, queue_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Gets a queue from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue was created.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Queue

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueuesListOperator(location, project_id=None, results_filter=None, page_size=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Lists queues from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queues were created.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • results_filter (str) – (Optional) Filter used to specify a subset of queues.

  • page_size (int) – (Optional) The maximum number of resources contained in the underlying API response.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

list[google.cloud.tasks_v2.types.Queue]

template_fields = ['location', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueueDeleteOperator(location, queue_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Deletes a queue from Cloud Tasks, even if it has tasks in it.

Parameters
  • location (str) – The location name in which the queue will be deleted.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueuePurgeOperator(location, queue_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Purges a queue by deleting all of its tasks from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be purged.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

list[google.cloud.tasks_v2.types.Queue]

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueuePauseOperator(location, queue_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Pauses a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be paused.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

list[google.cloud.tasks_v2.types.Queue]

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksQueueResumeOperator(location, queue_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Resumes a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be resumed.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

list[google.cloud.tasks_v2.types.Queue]

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksTaskCreateOperator(location, queue_name, task, project_id=None, task_name=None, response_view=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Creates a task in Cloud Tasks.

Parameters
  • location (str) – The location name in which the task will be created.

  • queue_name (str) – The queue’s name.

  • task (dict or class google.cloud.tasks_v2.types.Task) – The task to add. If a dict is provided, it must be of the same form as the protobuf message Task.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • task_name (str) – (Optional) The task’s name. If provided, it will be used to construct the full task path.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Task

template_fields = ['task', 'project_id', 'location', 'queue_name', 'task_name', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksTaskGetOperator(location, queue_name, task_name, project_id=None, response_view=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Gets a task from Cloud Tasks.

Parameters
  • location (str) – The location name in which the task was created.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Task

template_fields = ['location', 'queue_name', 'task_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksTasksListOperator(location, queue_name, project_id=None, response_view=None, page_size=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Lists the tasks in Cloud Tasks.

Parameters
  • location (str) – The location name in which the tasks were created.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • page_size (int) – (Optional) The maximum number of resources contained in the underlying API response.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

list[google.cloud.tasks_v2.types.Task]

template_fields = ['location', 'queue_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksTaskDeleteOperator(location, queue_name, task_name, project_id=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Deletes a task from Cloud Tasks.

Parameters
  • location (str) – The location name in which the task will be deleted.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

template_fields = ['location', 'queue_name', 'task_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]
class airflow.gcp.operators.tasks.CloudTasksTaskRunOperator(location, queue_name, task_name, project_id=None, response_view=None, retry=None, timeout=None, metadata=None, gcp_conn_id='google_cloud_default', *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Forces to run a task in Cloud Tasks.

Parameters
  • location (str) – The location name in which the task was created.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

  • gcp_conn_id (str) – (Optional) The connection ID used to connect to Google Cloud Platform.

Return type

google.cloud.tasks_v2.types.Task

template_fields = ['location', 'queue_name', 'task_name', 'project_id', 'gcp_conn_id'][source]
execute(self, context)[source]