backpressure

In akka-http, you can: Set akka.http.server.max-connections, which will prevent more than that number of connections. Exceeding this limit means clients will get connection timeouts. Set akka.http.server.pipelining-limit, which prevents a given connection from having more than this number of requests outstanding at once. Exceeding this means clients will get socket timeouts. These are both forms of backpressure from the http server to the client, but both are very low level, and only indirectly related to your server's performance. What seems better would be to backpressure at the http level,

I have a situation in Kafka where the producer publishes the messages at a very higher rate than the consumer consumption rate. I have to implement the back pressure implementation in kafka for further consumption and processing. Please let me know how can I implement in spark and also in normal java api. Kafka acts as the regulator here. You produce at whatever rate you want to into Kafka, scaling the brokers out to accommodate the ingest rate. You then consume as you want to; Kafka persists the data and tracks the offset of the consumers as they work their way through the data they read. You

Using google cloud functions, is there a way to manage execution concurrency the way AWS Lambda is doing? ( https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html ) My intent is to design a function that consumes a file of tasks and publish those tasks to a work queue (pub/sub). I want to have a function that consumes tasks from the work queue (pub/sub) and execute the task. The above could result in a large number of almost concurrent execution. My dowstream consumer service is slow and cannot consume many concurrent requests at a time. In all likelyhood, it would return HTTP