How to make it so that I can execute say 10 promises at a time in javascript to prevent rate limits on api calls?

Question

I have 1000 records that need to hit an API endpoint that is rate limited. I want to make it so that there is only 5 calls on the URL at any given time so that I am not making 1

Answer 1

Use waterfall pattern without a library and use a race condition to resolve on each iteration with reduce. And you can limit the number of calls by specifying the length of the array in Array.from.

var promise = Array.from({ length: 5 }).reduce(function (acc) {
  return acc.then(function (res) {
    return run().then(function (result) {
      res.push(result);
      return res;
    });
  });
}, Promise.resolve([]));


var guid = 0;
function run() {
  guid++;
  var id = guid;
  return new Promise(resolve => {
    // resolve in a random amount of time
    setTimeout(function () {
      console.log(id);
      resolve(id);
    }, (Math.random() * 1.5 | 0) * 1000);
  });
}

Answer 2

To limit the number of concurrent requests that are in-flight at once, I'd recommend using Bluebird's Promise.map() which offers a concurrency option. It will do all of the following for you:

1. Iterate your array
2. Limit the number of concurrent requests to whatever you set the concurrency option to
3. Collect all the results in order in the final results array

Here's how you would use it:

const Promise = require('bluebird');

Promise.map(records, r => {
    let placeName = r['Place Name'];
    return geocoder.geocodeAsync(placeName));
}, {concurrency: 5}).then(results => {
    // all results here
}).catch(err => {
    // process error here
});

Note: Rate limiting is not usually strictly the same as number of concurrent requests. Limiting the number of concurrent requests will make it more likely that you stay under a rate limit, but won't guarantee it. There are specific rate limiting modules that can manage to a rate limit more directly.

---

You can add a delay to each request using Bluebird's .delay().

const Promise = require('bluebird');

Promise.map(records, r => {
    let placeName = r['Place Name'];
    return geocoder.geocodeAsync(placeName)).delay(500);
}, {concurrency: 5}).then(results => {
    // all results here
}).catch(err => {
    // process error here
});

---

A classic algorithm for dealing with some types of rate limits is called the leaky bucket algorithm.

---

If your limit is 50 requests/sec, then you can just make sure that your concurrency number times your delay value never allows more than 50/sec.