Running multiple instances of nodejs server for scaling

Question

I am running a nodejs server on port 8080, so my server can only process one request at a time. I can see that if i send multiple requests in one single shot, new requests are q

Answer 1

First off, make sure your node.js process is ONLY using asynchronous I/O. If it's not compute intensive and using asynchronous I/O, it should be able to have many different requests "in-flight" at the same time. The design of node.js is particularly good at this if your code is designed properly. If you show us the crux of what it is doing on one of these requests, we can advise more specifically on whether your server code is designed properly for best throughput.

Second, instrument and measure, measure, measure. Understand where your bottlenecks are in your existing node.js server and what is causing the delay or sequencing you see. Sometimes there are ways to dramatically fix/improve your bottlenecks before you start adding lots more clusters or servers.

Third, use the node.js cluster module. This will create one master node.js process that automatically balances between several child processes. You generally want to creates a cluster child for each actual CPU you have in your server computer since that will get you the most use out of your CPU.

Fourth, if you need to scale to the point of multiple actual server computers, then you would use either a load balancer or reverse proxy such as nginx to share the load among multiple hosts. If you had a quad core CPUs in your server, you could run a cluster with four node.js processes on it on each server computer and then use nginx to balance among the several server boxes you had.

Note that adding multiple hosts that are load balanced by nginx is the last option here, not the first option.

Answer 2

Just a point to be added here over @sheplu, the pm2 module uses the node cluster module under the hood. But even then, pm2 is a very good choice, as it provides various other abstractions other than node cluster.

More info on it here: https://pm2.keymetrics.io/docs/usage/pm2-doc-single-page/

Answer 3

Like @poke said, you would use a reverse proxy and/or a load balancer in front.

But if you want a software to run multiple instances of node, with balancing and other stuffs, you should check pm2

http://pm2.keymetrics.io/