Cupy freeing unified memory

问题

I have a problem with freeing allocated memory in cupy. Due to memory constraints, I want to use unified memory. When I create a variable that will be allocated to the unified memory and want to free it, it is labelled as being freed and that the pool is now empty, to be used again, but when I take a look at a resource monitor, the memory is still not freed. When I create another variable it also adds to memory (I thought that perhaps the memory labelled as taken would be reused as is mentioned in the documentation but that is not the case.)

Here is a little program to test this with added sleep to be able to see the memory change in a resource monitor.

import cupy as cp
import time

def pool_stats(mempool):
    print('used:',mempool.used_bytes(),'bytes')
    print('total:',mempool.total_bytes(),'bytes\n')

pool = cp.cuda.MemoryPool(cp.cuda.memory.malloc_managed) # get unified pool
cp.cuda.set_allocator(pool.malloc) # set unified pool as default allocator

print('create first variable')
val1 = cp.zeros((50*1024,10*1024))
pool_stats(pool)
time.sleep(3)


print('delete first variable')
del val1
pool_stats(pool)
time.sleep(3)

print('free cupy memory')
pool.free_all_blocks()
pool_stats(pool)
time.sleep(3)

print('create second variable')
val2 = cp.zeros((50*1024,10*1024))
pool_stats(pool)
time.sleep(3)

print('delete second variable')
del val2
pool_stats(pool)
time.sleep(3)

print('free cupy memory')
pool.free_all_blocks()
pool_stats(pool)
time.sleep(3)

Here's the program's output:

create first variable
used: 4194304000 bytes
total: 4194304000 bytes

delete first variable
used: 0 bytes
total: 4194304000 bytes

free cupy memory
used: 0 bytes
total: 0 bytes

create second variable
used: 4194304000 bytes
total: 4194304000 bytes

delete second variable
used: 0 bytes
total: 4194304000 bytes

free cupy memory
used: 0 bytes
total: 0 bytes

So the output is what I'd expect. But this is not reflected by memory usage on resource monitors (nvtop and htop).

Here is what I got running this program. As seen with nvtop. Also, when the gpu memory runs out of space, it uses systems memory (as it is supposed to with unified memory) and this is also seen in htop(I'm trying to say I don't think its the hardware monitor's problem since it is seen across 2 different monitors)

Unified memory should behave like default memory.

The default memory graph is taken from practically the same program but without unified memory. I get the same console output across both too.

I also tried freeing pinned memory.

Is there something I'm doing wrong? Might this be a bug? is this perhaps a memory leak?

I also referred to this but couldn't find anything.

回答1:

From a now moderator deleted answer, this appears to be a bug in CuPy. A developer has raised a ticket on the github site, and a patch has been proposed (as at 2020-9-22).

In the short term, you will have to accept this as a problem and wait until a fix appears in a future release. If this is critical, then consider pulling the current development branch from github and building your own package. There is no user code workaround which can solve this.

来源：https://stackoverflow.com/questions/63998616/cupy-freeing-unified-memory