Malloc doesn't have to return memory to the operating system. Most malloc implementations on unix-like systems don't do it. Especially for smaller object sizes.
This is done for performance reasons.
I just noticed that this is potentially unclear. By "malloc" I mean the whole subsystem related to the malloc function - malloc, free, realloc, calloc and whatever special functions your libc might implement.