How to have heavy processing operations done in node.js

Question

I have a heavy data processing operation that I need to get done per 10-12 simulatenous request. I have read that for higher level of concurrency Node.js is a good platform and

Answer 1

When a heavy piece of computation is done in the callback, the event loop would be blocked until the computation is done. That means the callback will block the event loop for the 5 seconds.

My solution

It's possible to use a generator function to yield back control to the event loop. I will use a while loop that will run for 3 seconds to act as a long running callback.

Without a Generator function

let start = Date.now();

setInterval(() => console.log('resumed'), 500);
function loop() {
    while ((Date.now() - start) < 3000) { //while the difference between Date.now() and start is less than 3 seconds
        console.log('blocked')
    }
}

loop();

The output would be:

// blocked
// blocked
//
//  ... would not return to the event loop while the loop is running
//
// blocked
//...when the loop is over then the setInterval kicks in
// resumed
// resumed

With a Generator function

let gen;
let start = Date.now();

setInterval(() => console.log('resumed'), 500);

function *loop() {
    while ((Date.now() - start) < 3000) { //while the difference between Date.now() and start is less than 3 seconds
        console.log(yield output())
    }
}

function output() {
    setTimeout(() => gen.next('blocked'), 500)
}

gen = loop();
gen.next();

The output is:

// resumed
// blocked
//...returns control back to the event loop while though the loop is still running
// resumed
// blocked
//...end of the loop
// resumed
// resumed
// resumed

Using javascript generators can help run heavy computational functions that would yield back control to the event loop while it's still computing.

To know more about the event loop visit https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/function*

https://davidwalsh.name/es6-generators