问题
I am trying to create a Cloud Composer DAG to be triggered via a Pub/Sub message. There is the following example from Google which triggers a DAG every time a change occurs in a Cloud Storage bucket: https://cloud.google.com/composer/docs/how-to/using/triggering-with-gcf
However, on the beginning they say you can trigger DAGs in response to events, such as a change in a Cloud Storage bucket or a message pushed to Cloud Pub/Sub. I have spent a lot of time try to figure out how that can be done, but no result.
Can you please help or giving me some directions? Thanks!
回答1:
There are 2 ways to trigger a DAG by a Pub/Sub events.
- You can place a PubSubPullSensor at the beginning of your DAG. Your DAG will be triggered every time a Pub/Sub message can be pulled by the PubSubPullSensor. And it will execute the rest of the tasks in your DAG.
- You can also create a Cloud Function that acts as Pub/Sub trigger. And put the Composer DAG triggering logic inside the Cloud Function trigger. When a message is published to the Pub/Sub topic, the Cloud Function should be able to trigger the Composer DAG.
回答2:
To extend the public documentation page you already posted, you can configure a Cloud Function to run each time a message is published to a Cloud Pub/Sub topic. There is more information about that in another public documentation page.
To attach a function to a topic, set the --trigger-topic flag when deploying the function:
gcloud functions deploy $FUNCTION_NAME --runtime $RUNTIME --trigger-topic $TOPIC_NAME
来源：https://stackoverflow.com/questions/58551125/trigger-cloud-composer-dag-with-a-pub-sub-message