waiting for web worker making http call to finish

问题

I'm creating multiple web workers making http calls. I have to limit the number of web workers and so I'm trying to wait for some of the workers to finish.

here's an example of what I thought might work using Promises:

anArray.map(async contact => {
    await new Promise((res, rej) => {
          const worker = new Worker('./http.worker', { type: 'module' });
          worker.onmessage = () => {
            res();
          };
          worker.postMessage(contact);
    });
});

I thought this would wait for each promise to resolve before moving on to the next item... but it's not.

What could I do to make this work? or... I've also thought of building an array of workers and run a recursive loop that checks/waits for one to be available... I'm open to general suggestions for solving this.

回答1:

.map() is not promise aware. It does not look at the return value from each iteration to see if it's a promise and then pause the loop. Instead, it just blindly runs all the iterations one after another. When you return promises from .map() which you are with the async callback, that just means that your .map() will produce an array of promises and all your loop iterations will be "in-flight" at the same time, not sequenced.

If you want to iterate a loop and pause the loop in each iteration until a promise resolves, then use a regular for loop:

async function someFunc() {
    for (let contact of anArray) {
        await new Promise((res, rej) => {
              const worker = new Worker('./http.worker', { type: 'module' });
              worker.onmessage = () => {
                res();
              };
              worker.postMessage(contact);
        });
    }
}

FYI, http calls in Javascript are non-blocking and asynchronous so it's not entirely clear why you're doing them in WebWorkers. Unless you have CPU-intensive processing of the result, you can do the http requests in the main thread just fine without blocking it.

Also FYI, for a number of different options for processing an array, with only N requests in flight at the same time (where you decide what value N is), see these various answers:

runN(fn, limit, cnt, options): Loop through an API on multiple requests

pMap(array, fn, limit): Make several requests to an api that can only handle 20 at a time

rateLimitMap(array, requestsPerSec, maxInFlight, fn): Proper async method for max requests per second

mapConcurrent(array, maxConcurrent, fn): Promise.all() consumes all my ram

There are also features to do this built into the Bluebird promise library and the Async-promises library.

来源：https://stackoverflow.com/questions/60443116/waiting-for-web-worker-making-http-call-to-finish

标签

promise

async-await

web-worker