How to implement distributed rate limiter?

Question

Let\'s say, I have P processes running some business logic on N physical machines. These processes call some web service S, say. I want to ensure that not more than X calls

Answer 1

https://github.com/jdwyah/ratelimit-java provides distributed rate limits that should do just this. You can configure your limit as S per second / minute etc and choose burst size / refill rate of the leaky bucket that is under the covers.

Answer 2

Bucket4j is java implementation of "token-bucket" rate limiting algorithm. It works both locally and distributed(on top of JCache). For distributed use case you are free to choose any JCache implementation like Hazelcast or Apache Ignite. See this example of using Bucket4j in cluster.

Answer 3

I have been working on an opensource solution for these kind of problems.

Limitd is a "server" for limits. The limits are implemented using the Token Bucket Algorithm.

Basically you define limits in the service configuration like this:

buckets:
  "request to service a":
     per_minute: 10
  "request to service b":
     per_minute: 5

The service is run as a daemon listening on a TCP/IP port.

Then your application does something along these lines:

var limitd = new Limitd('limitd://my-limitd-address');

limitd.take('request to service a', 'app1' 1, function (err, result) {
  if (result.conformant) {
    console.log('everything is okay - this should be allowed');
  } else {
    console.error('too many calls to this thing');
  }
});

We are currently using this for rate-limiting and debouncing some application events.

The server is on:

https://github.com/auth0/limitd

We are planning to work on several SDKs but for now we only have node.js and partially implemented go:

https://github.com/limitd