airflow.gcp.sensors.pubsub
¶
This module contains a Google PubSub sensor.
Module Contents¶
-
class
airflow.gcp.sensors.pubsub.
PubSubPullSensor
(project, subscription, max_messages=5, return_immediately=False, ack_messages=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)[source]¶ Bases:
airflow.sensors.base_sensor_operator.BaseSensorOperator
Pulls messages from a PubSub subscription and passes them through XCom.
This sensor operator will pull up to
max_messages
messages from the specified PubSub subscription. When the subscription returns messages, the poke method’s criteria will be fulfilled and the messages will be returned from the operator and passed through XCom for downstream tasks.If
ack_messages
is set to True, messages will be immediately acknowledged before being returned, otherwise, downstream tasks will be responsible for acknowledging them.project
andsubscription
are templated so you can use variables in them.- Parameters
project (str) – the GCP project ID for the subscription (templated)
subscription (str) – the Pub/Sub subscription name. Do not include the full subscription path.
max_messages (int) – The maximum number of messages to retrieve per PubSub pull request
return_immediately (bool) – If True, instruct the PubSub API to return immediately if no messages are available for delivery.
ack_messages (bool) – If True, each message will be acknowledged immediately rather than by any downstream tasks
gcp_conn_id (str) – The connection ID to use connecting to Google Cloud Platform.
delegate_to (str) – The account to impersonate, if any. For this to work, the service account making the request must have domain-wide delegation enabled.