How to increase the number of containers in nodemanager in YARN

Question

A node in my YARN cluster has 64GB memory and 24 cores. I set the following properties in the yarn-site.xml:

  yarn.nodemanager.reso         


        

Answer 1

You need to tell YARN how to break down the memory to containers so for instance if you set the memory per container to 2GB will give you 16 containers

yarn.scheduler.minimum-allocation-mb
2048