google-cloud-pubsub

问题 I'm trying to implement a service which consumes a google pubsub subscription at its own pace. By that, I mean I need fine control on when I need to consume messages i.e get a batch of messages, pause for a while, do not get more than X messages... Using google client libraries I did not find a way to do it as the MessageReceiver is running in its own thread and I don't have any control on what exactly happens. Basically, being able to consume messages in a synchronous way should solve my

问题 I'm trying to implement a service which consumes a google pubsub subscription at its own pace. By that, I mean I need fine control on when I need to consume messages i.e get a batch of messages, pause for a while, do not get more than X messages... Using google client libraries I did not find a way to do it as the MessageReceiver is running in its own thread and I don't have any control on what exactly happens. Basically, being able to consume messages in a synchronous way should solve my

问题 I'm using Spark Dstream to pull and process data from Google PubSub. I'm looking for a way to move to structured streaming but still using Pub/Sub. Also, I should mention that my messages are Snappy compressed in Pub/Sub. I found this issue which claims that using Pub/Sub with structured streaming is not supported. Is someone has encountered this problem? Is it possible to implement custom Receiver to read the data from Pub/Sub Thanks 回答1: The feature request you referenced is still accurate:

问题 I'm trying to setup a flask endpoint so that whenever it receives a Gmail push notifications from PubSub, it sends a post request to another API/endpoint. However upon running the server, the process runs infinitely many times i.e. (receive notification, sends POST request to another API, returns 200 to PubSub) import Flask import requests app = Flask(__name__) # my endpoint receiving notifications from PubSub @app.route('/pubsub_notification_receiver') def receiver(): if request.method ==

问题 I'm trying to setup a flask endpoint so that whenever it receives a Gmail push notifications from PubSub, it sends a post request to another API/endpoint. However upon running the server, the process runs infinitely many times i.e. (receive notification, sends POST request to another API, returns 200 to PubSub) import Flask import requests app = Flask(__name__) # my endpoint receiving notifications from PubSub @app.route('/pubsub_notification_receiver') def receiver(): if request.method ==

问题 I'm trying to setup a flask endpoint so that whenever it receives a Gmail push notifications from PubSub, it sends a post request to another API/endpoint. However upon running the server, the process runs infinitely many times i.e. (receive notification, sends POST request to another API, returns 200 to PubSub) import Flask import requests app = Flask(__name__) # my endpoint receiving notifications from PubSub @app.route('/pubsub_notification_receiver') def receiver(): if request.method ==

问题 Closed . This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 11 months ago . Improve this question I have not worked on kafka much but wanted to build data pipeline in GCE. So we wanted to know Kafka vs PUB/Sub. Basically I want to know how message consistency, message availability, message reliability is maintained in both Kafka and Pub/sub Thanks 回答1:

问题 Closed . This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 11 months ago . Improve this question I have not worked on kafka much but wanted to build data pipeline in GCE. So we wanted to know Kafka vs PUB/Sub. Basically I want to know how message consistency, message availability, message reliability is maintained in both Kafka and Pub/sub Thanks 回答1:

问题 I am trying to use a pupsub emulator. It starts but when I try to use my python script I get the following error ModuleNotFoundError: No module named 'google' So i try to install the module. RUN pip install google-cloud-pubsub error ERROR: Command errored out with exit status 1: command: /usr/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-2hyoy1ly/grpcio/setup.py'"'"'; __file__='"'"'/tmp/pip-install-2hyoy1ly/grpcio/setup.py'"'"';f=getattr(tokenize, '"'

问题 I am working on cloud functions especially schedule functions. I need to trigger a function periodically each 5 minutes, but in only test step. I need to run it on pubsub emulator without deploying it. How to do it? I tried to use firebase shell, but it triggered only once exports.scheduledFunctionPlainEnglish =functions.pubsub.schedule('every 2 minutes') .onRun((context) => { functions.logger.log("this runs every 2 minutes") return null; }) 回答1: This is currently not supported for scheduled