Proper value of ES_HEAP_SIZE for a dedicated machine with two nodes in a cluster

Question

I have a dedicated machine for ES 2.2.0. It runs on a Windows Server 2012 R2 and has 128GB memory. Regarding ES, I plan to have TWO nodes in a cluster on this machine. According

Answer 1

The best practice is to never give more than 32GB of RAM to Elasticsearch (31GB to play safe) and leave the rest for the file system cache and the OS. Since Lucene doesn't use much heap, it will use most of what's left available to manage its segments.

The reason for this 32GB limit is that Java can use compressed pointers only if you don't allocate more than 32GB of RAM, if there's more RAM Java will go back to using normal pointers and the CPU-memory bandwidth performance will decrease even though you have more memory.

So, since you have a big machine with plenty of memory, what you can do is run two nodes on it with 31GB allocated to each node, which leaves 66GB for the file system cache, which Lucene will happily gobble up to do its things.

Now, just know that if you run your cluster on a single machine, and that machine goes down, your cluster is toast, and so will your client apps. However, if you have two such big machines (which seems to be the case), then make sure to set cluster.routing.allocation.same_shard.host: true in order to make sure that the primary and replica shards do not land on the same host, since you would not benefit from high availability.

More info:

• Read why ES decided to use compressed oops by default
• Read the details behind the compressed oops enhancement