why is my c program suddenly using 30g of virtual memory?

Question

In top, I noticed that my c program (using CUDA 3.2) has a virtual size of 28g or more (looking at VIRT), on every run right from the beginning. This doesn\'t make ANY sense to

Answer 1

UVA requires CUDA to allocate enough virtual memory to map all of both GPU and system memory. Please see post #5 in the following thread on the NVIDIA forums:

Answer 2

These two regions would be the culprit:

200000000-900000000 ---p 00000000 00:00 0
Size:           29360128 kB
Rss:                   0 kB
Pss:                   0 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:            0 kB
Anonymous:             0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
7f2e9deec000-7f2f131ec000 rw-s 33cc0c000 00:05 12626                     /dev/nvidia0
Size:            1920000 kB
Rss:             1920000 kB
Pss:             1920000 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:   1920000 kB
Referenced:      1920000 kB
Anonymous:             0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB

The first segment is a 30GB anonymous private segment, with no access to it allowed, mapped from 0x200000000-0x900000000. A bit mysterious, indeed - probably something to do with the nvidia driver's internal workings (maybe it wants to prevent allocations with those specific addresses?). It's not actually occupying any memory though - Rss is zero, and the access flags (---p) are set to deny all access, so (at the moment) actually allocating any memory to it won't happen. It's just a reserved section in your address space.

The other bit is the /dev/nvidia0 mapping, of two gigabytes. This is likely a direct mapping of part of the video card's RAM. It's not occupying memory as such - it's just reserving part of your address space to use to communicate with hardware.

So it's not really something to worry about. If you want to know how much memory you're really using, add up the Rss figures for all other memory segments (use the Private_* entries instead if you want to skip shared libraries and such).