Node.js memory consumption OS X vs Linux

问题

I have written a super-simple TCP/IP socket server in Node.js:

'use strict';

var net = require('net');

net.createServer(function (socket) {
  socket.resume();
  socket.once('end', function () {
    socket.removeAllListeners();
  });
}).listen(3000);

The once('end', ... should not even be necessary, but I wanted to be sure. Then I run the following command from the terminal to send 500 MBytes of data to the server:

$ cat 500mb.txt | nc localhost 3000

Now it gets interesting. I am watching the node process using top, and I do this on OS X and on Linux.

When I start Node.js…

OS X reports MEM 5152K (which seems fine)
Linux reports RES 15180 (which I interpret as 15 MB, but I'm not sure with this).

First question: Are both values comparable, or am I missing something here?

Then, as soon as I run cat ... | nc ..., memory usage raises. After one call…

OS X reports MEM 20M
Linux reports RES 92320 (which would means 92 (!) MB memory usage)

What is going on here? Why is Node.js using so much more memory on Linux than on OS X? Is my setup wrong? Am I missing something? ...?

回答1:

The RES value in top could be retrieved inside Node with process.memoryUsage() and is then in the rss value. Found this GitHub repo https://github.com/baryshev/rss-memory-leak and the corresponding Node issue https://github.com/joyent/node/issues/4217.

回答2:

The memory managers of Linux and Mac OS X are complex beasts, and working out the memory usage of a process is a black art.

On Linux, the RSS size is the resident process size. It includes things like mapped binaries and libraries and anonymously mapped pages (malloc data), but doesn't include unmapped data, such as parts of the binary that are not currently mapped, anonymous pages that have been mapped out, or lazily allocated anonymous pages. It will grow when there's plenty of memory even if your application is not doing much, and will shrink if there's memory pressure due to a different process. It's pretty much useless from a programmer's point of view (but very useful from the system administrators point of view).

On Mac OS X, the MEM column is documented as "Internal memory size". Without checking the sources of both top and the kernel, it is impossible to know what it reflects exactly.

In summary, the different values being displayed are probably nothing to worry about as long as your application is running fine. While looking at the OS-provided values can be useful as a rough idea of whether anything is wrong, you'll probably be way better served by looking at the values provided by V8's heap profiler.

回答3:

You may be experiencing Mavericks Memory Compression. (See the last point on this page)

In essence, Mac OSX will compress pages it deems "inactive" to save space. The algorithm the OS uses to determine "inactiveness" is probably secret (although I cannot prove it is). The compression algorithim Mavericks uses is WKdm.

If this indeed the case, OSX is probably deciding that certain pages v8 owns haven't been used in a while, and compressing them to make space for other programs. OSX will do this even if you have free memory, because it uses the memory not explicitly in use as a cache for program code and data that you might use in the future (which is also compressed).

I would also not that this is not exclusive to the other things people will inevitably bring up; differences in allocators and GCs, v8 quirks etc. count too, but this certainly matters too.

来源：https://stackoverflow.com/questions/26377566/node-js-memory-consumption-os-x-vs-linux

标签

Linux

node.js

macos

memory-consumption