What “deleted” means in /proc/$pid/maps?

问题

I downloaded libhugetlbfs.so and have a simple test source :

int glbarr[1024*1024]={0} ;
int main()
{
    char * ptr ;
    ptr = (char *) malloc( 1024 * 1024 * 1 ) ;
    printf(" press any key to go on \n");
    getchar() ;
    for(int idx=0;idx<100;idx++){
        char strtmp[64] = {0} ;
        sprintf(strtmp,"%020d",idx) ;
        strcpy( ptr+1024*idx , strtmp ) ;
    } //for 
    for(int idx=0;idx<100;idx++){
        glbarr[idx] = idx ;
    }
    printf(" press any key to go on \n");
    getchar() ;
} // main

then set the env :

export LD_PRELOAD=libhugetlbfs.so
export HUGETLB_MORECORE=yes
export HUGETLB_DEBUG=1

At last , execute test_malloc.exe :

 INFO: Found pagesize 2048 kB
 INFO: Detected page sizes:
 INFO:    Size: 2048 kB (default)  Mount: /mnt/SharedMem_2M
 INFO: Parsed kernel version: [3] . [10] . [0] 
 INFO: Feature private_reservations is present in this kernel
 INFO: Feature noreserve_safe is present in this kernel
 INFO: Feature map_hugetlb is present in this kernel
 INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
 INFO: Kernel supports MAP_HUGETLB
 INFO: HUGETLB_SHARE=0, sharing disabled
 INFO: HUGETLB_NO_RESERVE=no, reservations enabled
 INFO: Segment 0 (phdr 2): 0x400000-0x400a04  (filesz=0xa04) (prot = 0x5)
 INFO: Segment 1 (phdr 3): 0x600de0-0xa01080  (filesz=0x274) (prot = 0x3)
 DEBUG: symbol to copy at 0x601060: stdin
 DEBUG: Total memsz = 0x400ca4, memsz of largest segment = 0x4002a0
 INFO: libhugetlbfs version: 2.20
 INFO: Mapped hugeseg at 0x2aaaaac00000. Copying 0xa04 bytes and 0 extra bytes from 0x400000...done
 INFO: Prepare succeeded
 INFO: Mapped hugeseg at 0x2aaaaac00000. Copying 0x274 bytes and 0x14 extra bytes from 0x600de0...done
 INFO: Prepare succeeded
 INFO: setup_morecore(): heapaddr = 0x1c00000
 INFO: hugetlbfs_morecore(2101248) = ...
 INFO: heapbase = 0x1c00000, heaptop = 0x1c00000, mapsize = 0, delta=2101248
 INFO: Attempting to map 4194304 bytes
 INFO: ... = 0x1c00000
 INFO: hugetlbfs_morecore(0) = ...
 INFO: heapbase = 0x1c00000, heaptop = 0x1e01000, mapsize = 400000, delta=-2093056
 INFO: ... = 0x1e01000

and the /proc/pidof test_malloc.exe/maps :

00400000-00600000 r-xp 00000000 00:2b 6019488                            /mnt/SharedMem_2M/libhugetlbfs.tmp.uI55WD (deleted)
00600000-00c00000 rw-p 00000000 00:2b 6123885                            /mnt/SharedMem_2M/libhugetlbfs.tmp.VUALYM (deleted)
01c00000-02000000 rw-p 00000000 00:0d 6123886                            /anon_hugepage (deleted)

numastat -m showes hugepage really in use for 8M , What bother me is what do "deleted" means in maps output ?! either in /mnt/SharedMem_2M or /anon_hugepage

Edit :

and the debug information :

INFO: Found pagesize 2048 kB
INFO: Detected page sizes:
INFO:    Size: 2048 kB (default)  Mount: /mnt/SharedMem_2M
INFO: Parsed kernel version: [3] . [10] . [0] 
INFO: Feature private_reservations is present in this kernel
INFO: Feature noreserve_safe is present in this kernel
INFO: Feature map_hugetlb is present in this kernel
INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
INFO: Kernel supports MAP_HUGETLB
INFO: HUGETLB_SHARE=0, sharing disabled
INFO: HUGETLB_NO_RESERVE=no, reservations enabled
INFO: Segment 0 (phdr 3): 0x600de0-0xa01080  (filesz=0x274) (prot = 0x3)
DEBUG: symbol to copy at 0x601060: stdin
DEBUG: Total memsz = 0x4002a0, memsz of largest segment = 0x4002a0
INFO: libhugetlbfs version: 2.20
INFO: Mapped hugeseg at 0x2aaaaac00000. Copying 0x274 bytes and 0x14 extra bytes from 0x600de0...done
INFO: Prepare succeeded
INFO: setup_morecore(): heapaddr = 0x2200000
INFO: hugetlbfs_morecore(2101248) = ...
INFO: heapbase = 0x2200000, heaptop = 0x2200000, mapsize = 0, delta=2101248
INFO: Attempting to map 4194304 bytes
INFO: ... = 0x2200000
INFO: hugetlbfs_morecore(0) = ...
INFO: heapbase = 0x2200000, heaptop = 0x2401000, mapsize = 400000, delta=-2093056
INFO: ... = 0x2401000

and the maps :

00400000 default file=/home/marschen/test/posix-memalign/test_malloc.exe mapped=1 N0=1 kernelpagesize_kB=4
00600000 default file=/mnt/SharedMem_2M/libhugetlbfs.tmp.85Y41e\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=2048
02200000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=2048

回答1:

Temporary file deletion (unlinking) is normal for libhugetlbfs when it uses hugetlb fs pseudo filesystem (grep hugetlbfs /proc/filesystems) for getting mmaps backed with hugetlb pages.

For example, there is hugetlbfs_unlinked_fd function of libhugetlbfs/hugeutils.c https://github.com/libhugetlbfs/libhugetlbfs/blob/e44180072b796c0e28e53c4d01ef6279caaa2a99/hugeutils.c#L1033

int hugetlbfs_unlinked_fd_for_size(long page_size)
{
    const char *path;
    char name[PATH_MAX+1];
    int fd;

    path = hugetlbfs_find_path_for_size(page_size);
    ..
    name[sizeof(name)-1] = '\0';

    strcpy(name, path);
    strncat(name, "/libhugetlbfs.tmp.XXXXXX", sizeof(name)-1);
    /* FIXME: deal with overflows */

    fd = mkstemp64(name);
    ....

    unlink(name);

    return fd;
}

Temporary file name is randomly generated in mkstemp function; it also creates the file and opens it. Then this file is unlinked (man 2 unlink) from the filesystem (file name is marked as deleted in the directory, there is still inode and file data, but other programs can't access this file by name).

While unlinked fd is opened, it can be used to work with hugetlb mmap and to store data. Only when this fd is closed, file data will be actually deleted by fs.

Early unlinking of mktemp files is often used: When a file created with mkstemp() is deleted?

Some useful information can be also listed in HOWTO of libhugetlbfs project https://github.com/libhugetlbfs/libhugetlbfs/blob/master/HOWTO

来源：https://stackoverflow.com/questions/50244427/what-deleted-means-in-proc-pid-maps

标签

linux-kernel

elf

huge-pages