how to keep variables that share all node processes in node cluster?

Question

It seems like all the node woker processes are working as if it is executing a new copy of the same application. But would like to keep some variables that are shared by all nod

Answer 1

You can try to communicate between master process and child processes. For example:

script test.master.js:

var cluster = require('cluster');
var childScript = __dirname + '/test.child.js';

cluster.setupMaster({ exec: childScript });

proc = cluster.fork();
proc.on('message', function(message) {
    console.log('message from child: ', message);
    proc.send('Hello from master!');
});

script test.child.js:

console.log('Child initializing..');

process.on('message', function(message) {
    console.log('message from master: ', message);
});

process.send('Hello from Child!');

Answer 2

I think the whole idea of cluster is having instances that can run independently on different cpus. Sharing memory (a global variable) that they both can access and change introduces more complexity (locks, etc) and makes these instances depend on one another.

An outside database would be a good solution to this since it takes care of all th data access problems, but it most likely lowers performance.

Messaging is a better idea. You can hold local instances of a var in your clusters. When a cluster updates the value, send a message to the rest of them and update the value. This is great because it's async and nonblocking but again your value update won't reflect immediately.

How about this: store vars on a db and everytime there's a value change notify the instances. They can store the new values in local vars and make db calls only when it's needed

Answer 3

I used external memcached or redis server for it.

Answer 4

All worker processes are indeed new copies of your application. Each worker is a full featured process created with child_process.spawn. So no, they don't share variables. And it's probably best this way. If you want to share information between worker processes (typically sessions) you should look into storing these information in a database.

If you're ready to go node all the way, you could use something like dnode to have your workers ask the master process for data.

Answer 5

Nobody has mentioned this yet but this is a perfect case for Node Worker Threads which just got out of experimental mode in the latest version of Node v11.11.0.

Answer 6

If you're looking to share things on a read-only basis, check out mmap-object. I use it for large in-memory lookup tables.

Checking just now on a server, a 346M file is taking up a total of 156M of memory (mmap only pages in what's accessed) and mmap-object sharing it among 44 processes for a per-process overhead of 3.5M.

Because it's read-only I don't have to worry about inter-process locking and the messiness that can bring along.