Erlang and its consumption of Heap Memory

Question

I have been running a highly concurrent application on my HP Proliant Servers. The application is a file system indexer i coded in erlang. It spawns a process per Folder it

Answer 1

I haven't had time to look at the source, but here are some comments:

Question 1. With such a powerful server, why would the operating system fail to provide such memory to the application (it was the only application running)?

Because the Erlang VM tried to consume more than the available free memory.

Question 2. The Erlang Emulator i start is instructed to be able to spawn as many processes as it may need. the value +P 13421779. Is Erlang VM failing to access this memory or failing to allocate it to its processes ?

No. If you would have run out of processess, the Erlang VM would have said so (and the VM would still be up and running):

=ERROR REPORT==== 18-Aug-2011::10:04:04 ===
Error in process <0.31775.138> with exit value: {system_limit,[{erlang,spawn_link,    [erlang,apply,[#Fun<shell.3.130303173>,[]]]},{erlang,spawn_link,1},{shell,get_command,5},    {shell,server_loop,7}]}

Question 3. To Solaris, it sees one process: epmd, perhaps containing and starting thousands of micro threads. What configurations can i make to Solaris to be able to never stop my application however much "memory hungry" it may be? Swap space available is 16 GB, RAM 20 GB, honestly, there must be something wrong.

epmd is the Erlang port mapping deamon. It's responsible for managing distributet Erlang and has nothing to with your individual Erlang application. The processes you should look for will be name beam.smp most likely. These will show the OS memory consumption of the Erlang VM etc.

Question 4. Which configurations can i make to the Erlang Emulator, to avoid these heap memory crash dumps especially when all the memory it may need is available on the server? How will i run more memory consuming apps on this server if Erlang still fails to allocate such memory to a simple file system indexer (well its heavily concurrent)?

The Erlang VM should be able to use all of the available memory in your machine. However, it depends on how your application is written. There can be many reasons for memory leaks:

• Atom table filling up (you create too many unique atoms)
• ETS or Mnesia tables are not garbage collected (you do not delete old unused elements)
• Not enough memory for processes (you spawn too many processess)
• Too many binaries are created (you might keep unused references to old binaries)