Malloc allocates memory more than RAM

Question

I just executed a program that mallocs 13 MB in a 12 MB machine (QEMU Emulated!) . Not just that, i even browsed through the memory and filled junk in it...

Answer 1

Unless the virtualized OS has swap available, what you're encountering is called overcommit, and it basically exists because the easy way to manage resources in a system with virtual memory and demand/copy-on-write pages is not to manage them. Overcommit is a lot like a bank loaning out more money than it actually has -- it seems to work for a while, then things come crashing down. The first thing you should do when setting up a Linux system is fix this with the command:

echo "2" > /proc/sys/vm/overcommit_memory

That just affects the currently running kernel; you can make it permanent by adding a line to /etc/sysctl.conf:

vm.overcommit_memory=2

Answer 2

It's called virtual memory which is allocated for your program. It's not real memory which you call RAM.

There is a max limit for virtual memory as well, but it's higher than RAM. It's implemented (and defined) by your operating system.

Answer 3

Sounds like your operating system is swapping pages:

Paging is an important part of virtual memory implementation in most contemporary general-purpose operating systems, allowing them to use disk storage for data that does not fit into physical random-access memory (RAM).

In other words, the operating system is using some of your hard disk space to satisfy your 13 MB allocation request (at great expense of speed, since the hard disk is much, much slower than RAM).

Answer 4

This is called as Lazy Allocation.

Most OS like Linux have an Lazy Allocation memory model wherein the returned memory address is a virtual address and the actual allocation only happens at access-time. The OS assumes that it will be able to provide this allocation at access-Time.

The memory allocated by malloc is not backed by real memory until the program actually touches it.

While, since calloc initializes the memory to 0 you can be assured that the OS has already backed the allocation with actual RAM (or swap).

Try using callocand most probably it will return you out of memory unless your swap file/partition is big enough to satisfy the request.