airflow.gcp.hooks.tasks

This module contains a CloudTasksHook which allows you to connect to GCP Cloud Tasks service, performing actions to queues or tasks.

Module Contents

class airflow.gcp.hooks.tasks.CloudTasksHook(gcp_conn_id='google_cloud_default', delegate_to=None)[source]

Bases: airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook

Hook for Google Cloud Tasks APIs. Cloud Tasks allows developers to manage the execution of background work in their applications.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

Parameters
  • gcp_conn_id (str) – The connection ID to use when fetching connection info.

  • delegate_to (str) – The account to impersonate, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

get_conn(self)[source]

Provides a client for interacting with the Cloud Tasks API.

Returns

GCP Cloud Tasks API Client

Return type

google.cloud.tasks_v2.CloudTasksClient

create_queue(self, location, task_queue, project_id=None, queue_name=None, retry=None, timeout=None, metadata=None)[source]

Creates a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be created.

  • task_queue (dict or class google.cloud.tasks_v2.types.Queue) – The task queue to create. Queue’s name cannot be the same as an existing queue. If a dict is provided, it must be of the same form as the protobuf message Queue.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • queue_name (str) – (Optional) The queue’s name. If provided, it will be used to construct the full queue path.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Queue

update_queue(self, task_queue, project_id=None, location=None, queue_name=None, update_mask=None, retry=None, timeout=None, metadata=None)[source]

Updates a queue in Cloud Tasks.

Parameters
  • task_queue (dict or class google.cloud.tasks_v2.types.Queue) – The task queue to update. This method creates the queue if it does not exist and updates the queue if it does exist. The queue’s name must be specified.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • location (str) – (Optional) The location name in which the queue will be updated. If provided, it will be used to construct the full queue path.

  • queue_name (str) – (Optional) The queue’s name. If provided, it will be used to construct the full queue path.

  • update_mask (dict or class google.cloud.tasks_v2.types.FieldMask) – A mast used to specify which fields of the queue are being updated. If empty, then all fields will be updated. If a dict is provided, it must be of the same form as the protobuf message.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Queue

get_queue(self, location, queue_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Gets a queue from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue was created.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Queue

list_queues(self, location, project_id=None, results_filter=None, page_size=None, retry=None, timeout=None, metadata=None)[source]

Lists queues from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queues were created.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • results_filter (str) – (Optional) Filter used to specify a subset of queues.

  • page_size (int) – (Optional) The maximum number of resources contained in the underlying API response.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

list[google.cloud.tasks_v2.types.Queue]

delete_queue(self, location, queue_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Deletes a queue from Cloud Tasks, even if it has tasks in it.

Parameters
  • location (str) – The location name in which the queue will be deleted.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

purge_queue(self, location, queue_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Purges a queue by deleting all of its tasks from Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be purged.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

list[google.cloud.tasks_v2.types.Queue]

pause_queue(self, location, queue_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Pauses a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be paused.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

list[google.cloud.tasks_v2.types.Queue]

resume_queue(self, location, queue_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Resumes a queue in Cloud Tasks.

Parameters
  • location (str) – The location name in which the queue will be resumed.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

list[google.cloud.tasks_v2.types.Queue]

create_task(self, location, queue_name, task, project_id=None, task_name=None, response_view=None, retry=None, timeout=None, metadata=None)[source]

Creates a task in Cloud Tasks.

Parameters
  • location (str) – The location name in which the task will be created.

  • queue_name (str) – The queue’s name.

  • task (dict or class google.cloud.tasks_v2.types.Task) – The task to add. If a dict is provided, it must be of the same form as the protobuf message Task.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • task_name (str) – (Optional) The task’s name. If provided, it will be used to construct the full task path.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Task

get_task(self, location, queue_name, task_name, project_id=None, response_view=None, retry=None, timeout=None, metadata=None)[source]

Gets a task from Cloud Tasks.

Parameters
  • location (str) – The location name in which the task was created.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Task

list_tasks(self, location, queue_name, project_id=None, response_view=None, page_size=None, retry=None, timeout=None, metadata=None)[source]

Lists the tasks in Cloud Tasks.

Parameters
  • location (str) – The location name in which the tasks were created.

  • queue_name (str) – The queue’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • page_size (int) – (Optional) The maximum number of resources contained in the underlying API response.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

list[google.cloud.tasks_v2.types.Task]

delete_task(self, location, queue_name, task_name, project_id=None, retry=None, timeout=None, metadata=None)[source]

Deletes a task from Cloud Tasks.

Parameters
  • location (str) – The location name in which the task will be deleted.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

run_task(self, location, queue_name, task_name, project_id=None, response_view=None, retry=None, timeout=None, metadata=None)[source]

Forces to run a task in Cloud Tasks.

Parameters
  • location (str) – The location name in which the task was created.

  • queue_name (str) – The queue’s name.

  • task_name (str) – The task’s name.

  • project_id (str) – (Optional) The ID of the GCP project that owns the Cloud Tasks. If set to None or missing, the default project_id from the GCP connection is used.

  • response_view (google.cloud.tasks_v2.types.Task.View) – (Optional) This field specifies which subset of the Task will be returned.

  • retry (google.api_core.retry.Retry) – (Optional) A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (float) – (Optional) The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (sequence[tuple[str, str]]]) – (Optional) Additional metadata that is provided to the method.

Return type

google.cloud.tasks_v2.types.Task