google-cloud-pubsub

问题 I am trying to stream PubSub message of type JSON to spanner database and the insert_update works very well. Spanner table has composite primary key, so need to delete the existing data before inserting new data from PubSub (so only latest data is present). Spanner replace or insert/update mutations does not work in this case. I added pipeline import org.apache.beam.* ; public class PubSubToSpannerPipeline { // JSON to TableData Object public static class PubSubToTableDataFn extends DoFn

问题 Can anyone give details on the Dynamic rate limiting implemented by the Pub/Sub system? I couldn't find any details on the gcloud docs or the faq pages. Here is my pubsub usage: I'm planning to use pubsub in our production. Right now, I have 1 topic, 1 subscription and 1 subscriber (Webhook HTTPS callback). Sometimes my subscriber can throw an exception (very rarely), in that situation my subscriber shall return a 400 response back to the pubsub, so that the pubsub can retain the message and

问题 I have a working streaming pipeline in apache beam [python] that ingests data from pub/sub, performs enrichment in dataflow and passes it to big-query. Withing the streaming window, I would like to ensure that messages are not getting duplicated, (as pub/sub guarantees only at least once delivery). So, I figured I'd just use the distinct method from beam, but as soon as I use it my pipeline breaks (can't proceed any further, any local prints are also not visible). Here is my pipeline code:

问题 We are building a serverless platform consisting of Android and iOS apps build using React Native and on the backend we use Google Cloud Functions and Firebase. Given that some actions are handled by multiple Cloud Functions, we thought it would be a good idea to have the apps Publish to a Pub/Sub topic and then have the different Cloud Functions fire Subscribe events when the apps publish. Is it good practice to have a React Native app Publish directly to a Google Pub/Sub topic or should we

问题 We are building a serverless platform consisting of Android and iOS apps build using React Native and on the backend we use Google Cloud Functions and Firebase. Given that some actions are handled by multiple Cloud Functions, we thought it would be a good idea to have the apps Publish to a Pub/Sub topic and then have the different Cloud Functions fire Subscribe events when the apps publish. Is it good practice to have a React Native app Publish directly to a Google Pub/Sub topic or should we

问题 I have a Firebase Function that subscribes to a Cloud PubSub topic. App is initialized very simply like this: import * as admin from 'firebase-admin'; admin.initializeApp(); I'm getting this error: "Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information. at GoogleAuth.getApplicationDefaultAsync (/srv/functions/node_modules/google-auth-library/build/src/auth/googleauth.js:161:19) at process._tickCallback

问题 I'm building a Dataflow pipeline that reads from pubsub and sends requests to a 3rd party API. The pipeline use THROUGHPUT_BASED autoscaling. However when I was doing a load test against it, after it autoscaled to 4 works to catch up with the backlog in pubsub, but it seems the same workload was spread out event between works, but overall throughput did not increase significantly. ^ Number of unacknowledged messages in pubsub. The peak is when traffic stopped going in ^ Bytes sent from each

问题 I'm building a Dataflow pipeline that reads from pubsub and sends requests to a 3rd party API. The pipeline use THROUGHPUT_BASED autoscaling. However when I was doing a load test against it, after it autoscaled to 4 works to catch up with the backlog in pubsub, but it seems the same workload was spread out event between works, but overall throughput did not increase significantly. ^ Number of unacknowledged messages in pubsub. The peak is when traffic stopped going in ^ Bytes sent from each

问题 I'm trying to implement a service which consumes a google pubsub subscription at its own pace. By that, I mean I need fine control on when I need to consume messages i.e get a batch of messages, pause for a while, do not get more than X messages... Using google client libraries I did not find a way to do it as the MessageReceiver is running in its own thread and I don't have any control on what exactly happens. Basically, being able to consume messages in a synchronous way should solve my

问题 I'm trying to implement a service which consumes a google pubsub subscription at its own pace. By that, I mean I need fine control on when I need to consume messages i.e get a batch of messages, pause for a while, do not get more than X messages... Using google client libraries I did not find a way to do it as the MessageReceiver is running in its own thread and I don't have any control on what exactly happens. Basically, being able to consume messages in a synchronous way should solve my