C Program on Linux to exhaust memory

Question

I would like to write a program to consume all the memory available to understand the outcome. I\'ve heard that linux starts killing the processes once it is unable to alloc

Answer 1

You should write to the allocated blocks. If you just ask for memory, linux might just hand out a reservation for memory, but nothing will be allocated until the memory is accessed.

int main()
{
        while(1)
        {
                void *m = malloc(1024*1024);
                memset(m,0,1024*1024);
        }
        return 0;
}

You really only need to write 1 byte on every page (4096 bytes on x86 normally) though.

Answer 2

A little known fact (though it is well documented) - you can (as root) prevent the OOM killer from claiming your process (or any other process) as one of its victims. Here is a snippet from something directly out of my editor, where I am (based on configuration data) locking all allocated memory to avoid being paged out and (optionally) telling the OOM killer not to bother me:

static int set_priority(nex_payload_t *p)
{
    struct sched_param sched;
    int maxpri, minpri;
    FILE *fp;
    int no_oom = -17;

    if (p->cfg.lock_memory)
        mlockall(MCL_CURRENT | MCL_FUTURE);

    if (p->cfg.prevent_oom) {
        fp = fopen("/proc/self/oom_adj", "w");
        if (fp) {
            /* Don't OOM me, Bro! */
            fprintf(fp, "%d", no_oom);
            fclose(fp);
        }
    }

I'm not showing what I'm doing with scheduler parameters as its not relevant to the question.

This will prevent the OOM killer from getting your process before it has a chance to produce the (in this case) desired effect. You will also, in effect, force most other processes to disk.

So, in short, to see fireworks really quickly...

1. Tell the OOM killer not to bother you
2. Lock your memory
3. Allocate and initialize (zero out) blocks in a never ending loop, or until malloc() fails

Be sure to look at ulimit as well, and run your tests as root.

The code I showed is part of a daemon that simply can not fail, it runs at a very high weight (selectively using the RR or FIFO scheduler) and can not (ever) be paged out.

Answer 3

I just exexuted @John La Rooy's snippet:

#include <stdlib.h>
#include <stdio.h>

int main()
{
  while(1)
  {
    printf("malloc %d\n", (int)malloc(1024*1024));
  }
  return 0;
}

but it exhausted my memory very fast and cause the system hanged so that I had to restart it.

So I recommend you change some code.

For example: On my ubuntu 16.04 LTS the code below takes about 1.5 GB ram, physical memory consumption raised from 1.8 GB to 3.3 GB after executing and go down back to 1.8GiB after finishing execution.Though it looks like I have allocate 300GiB ram in the code.

#include <stdlib.h>
#include <stdio.h>

int main()
{
  while(int i<300000)
  {
    printf("malloc %p\n", malloc(1024*1024));
    i += 1;
  }
  return 0;
}

When index i is less then 100000(ie, allocate less than 100 GB), either physical or virtual memory are just very slightly used(less then 100MB), I don't know why, may be there is something to do with virtual memory.

One thing interesting is that when the physical memory begins to shrink, the addresses malloc() returns definitely changes, see picture link below.

I used malloc() and calloc(), seems that they behave similarily in occupying physical memory.

memory address number changes from 48 bits to 28 bits when physical memory begins shrinking

Answer 4

Linux "over commits" memory. This means that physical memory is only given to a process when the process first tries to access it, not when the malloc is first executed. To disable this behavior, do the following (as root):

echo 2 > /proc/sys/vm/overcommit_memory

Then try running your program.

Answer 5

Have a look at this program. When there is no longer enough memory malloc starts returning 0

#include <stdlib.h>
#include <stdio.h>

int main()
{
  while(1)
  {
    printf("malloc %d\n", (int)malloc(1024*1024));
  }
  return 0;
}

Answer 6

On a 32-bit Linux system, the maximum that a single process can allocate in its address space is approximately 3Gb.

This means that it is unlikely that you'll exhaust the memory with a single process.

On the other hand, on 64-bit machine you can allocate as much as you like.

As others have noted, it is also necessary to initialise the memory otherwise it does not actually consume pages.

malloc will start giving an error if EITHER the OS has no virtual memory left OR the process is out of address space (or has insufficient to satisfy the requested allocation).

Linux's VM overcommit also affects exactly when this is and what happens, as others have noted.