How to determine CPU and memory consumption from inside a process?

Question

I once had the task of determining the following performance parameters from inside a running application:

• Total virtual memory available
• Virtual memo

Answer 1

Linux

In Linux, this information is available in the /proc file system. I'm not a big fan of the text file format used, as each Linux distribution seems to customize at least one important file. A quick look as the source to 'ps' reveals the mess.

But here is where to find the information you seek:

/proc/meminfo contains the majority of the system-wide information you seek. Here it looks like on my system; I think you are interested in MemTotal, MemFree, SwapTotal, and SwapFree:

Anderson cxc # more /proc/meminfo
MemTotal:      4083948 kB
MemFree:       2198520 kB
Buffers:         82080 kB
Cached:        1141460 kB
SwapCached:          0 kB
Active:        1137960 kB
Inactive:       608588 kB
HighTotal:     3276672 kB
HighFree:      1607744 kB
LowTotal:       807276 kB
LowFree:        590776 kB
SwapTotal:     2096440 kB
SwapFree:      2096440 kB
Dirty:              32 kB
Writeback:           0 kB
AnonPages:      523252 kB
Mapped:          93560 kB
Slab:            52880 kB
SReclaimable:    24652 kB
SUnreclaim:      28228 kB
PageTables:       2284 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   4138412 kB
Committed_AS:  1845072 kB
VmallocTotal:   118776 kB
VmallocUsed:      3964 kB
VmallocChunk:   112860 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

For CPU utilization, you have to do a little work. Linux makes available overall CPU utilization since system start; this probably isn't what you are interested in. If you want to know what the CPU utilization was for the last second, or 10 seconds, then you need to query the information and calculate it yourself.

The information is available in /proc/stat, which is documented pretty well at http://www.linuxhowtos.org/System/procstat.htm; here is what it looks like on my 4-core box:

Anderson cxc #  more /proc/stat
cpu  2329889 0 2364567 1063530460 9034 9463 96111 0
cpu0 572526 0 636532 265864398 2928 1621 6899 0
cpu1 590441 0 531079 265949732 4763 351 8522 0
cpu2 562983 0 645163 265796890 682 7490 71650 0
cpu3 603938 0 551790 265919440 660 0 9040 0
intr 37124247
ctxt 50795173133
btime 1218807985
processes 116889
procs_running 1
procs_blocked 0

First, you need to determine how many CPUs (or processors, or processing cores) are available in the system. To do this, count the number of 'cpuN' entries, where N starts at 0 and increments. Don't count the 'cpu' line, which is a combination of the cpuN lines. In my example, you can see cpu0 through cpu3, for a total of 4 processors. From now on, you can ignore cpu0..cpu3, and focus only on the 'cpu' line.

Next, you need to know that the fourth number in these lines is a measure of idle time, and thus the fourth number on the 'cpu' line is the total idle time for all processors since boot time. This time is measured in Linux "jiffies", which are 1/100 of a second each.

But you don't care about the total idle time; you care about the idle time in a given period, e.g., the last second. Do calculate that, you need to read this file twice, 1 second apart.Then you can do a diff of the fourth value of the line. For example, if you take a sample and get:

cpu  2330047 0 2365006 1063853632 9035 9463 96114 0

Then one second later you get this sample:

cpu  2330047 0 2365007 1063854028 9035 9463 96114 0

Subtract the two numbers, and you get a diff of 396, which means that your CPU had been idle for 3.96 seconds out of the last 1.00 second. The trick, of course, is that you need to divide by the number of processors. 3.96 / 4 = 0.99, and there is your idle percentage; 99% idle, and 1% busy.

In my code, I have a ring buffer of 360 entries, and I read this file every second. That lets me quickly calculate the CPU utilization for 1 second, 10 seconds, etc., all the way up to 1 hour.

For the process-specific information, you have to look in /proc/pid; if you don't care abut your pid, you can look in /proc/self.

CPU used by your process is available in /proc/self/stat. This is an odd-looking file consisting of a single line; for example:

19340 (whatever) S 19115 19115 3084 34816 19115 4202752 118200 607 0 0 770 384 2
 7 20 0 77 0 266764385 692477952 105074 4294967295 134512640 146462952 321468364
8 3214683328 4294960144 0 2147221247 268439552 1276 4294967295 0 0 17 0 0 0 0

The important data here are the 13th and 14th tokens (0 and 770 here). The 13th token is the number of jiffies that the process has executed in user mode, and the 14th is the number of jiffies that the process has executed in kernel mode. Add the two together, and you have its total CPU utilization.

Again, you will have to sample this file periodically, and calculate the diff, in order to determine the process's CPU usage over time.

Edit: remember that when you calculate your process's CPU utilization, you have to take into account 1) the number of threads in your process, and 2) the number of processors in the system. For example, if your single-threaded process is using only 25% of the CPU, that could be good or bad. Good on a single-processor system, but bad on a 4-processor system; this means that your process is running constantly, and using 100% of the CPU cycles available to it.

For the process-specific memory information, you ahve to look at /proc/self/status, which looks like this:

Name:   whatever
State:  S (sleeping)
Tgid:   19340
Pid:    19340
PPid:   19115
TracerPid:      0
Uid:    0       0       0       0
Gid:    0       0       0       0
FDSize: 256
Groups: 0 1 2 3 4 6 10 11 20 26 27
VmPeak:   676252 kB
VmSize:   651352 kB
VmLck:         0 kB
VmHWM:    420300 kB
VmRSS:    420296 kB
VmData:   581028 kB
VmStk:       112 kB
VmExe:     11672 kB
VmLib:     76608 kB
VmPTE:      1244 kB
Threads:        77
SigQ:   0/36864
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: fffffffe7ffbfeff
SigIgn: 0000000010001000
SigCgt: 20000001800004fc
CapInh: 0000000000000000
CapPrm: 00000000ffffffff
CapEff: 00000000fffffeff
Cpus_allowed:   0f
Mems_allowed:   1
voluntary_ctxt_switches:        6518
nonvoluntary_ctxt_switches:     6598

The entries that start with 'Vm' are the interesting ones:

• VmPeak is the maximum virtual memory space used by the process, in kB (1024 bytes).
• VmSize is the current virtual memory space used by the process, in kB. In my example, it's pretty large: 651,352 kB, or about 636 megabytes.
• VmRss is the amount of memory that have been mapped into the process' address space, or its resident set size. This is substantially smaller (420,296 kB, or about 410 megabytes). The difference: my program has mapped 636 MB via mmap(), but has only accessed 410 MB of it, and thus only 410 MB of pages have been assigned to it.

The only item I'm not sure about is Swapspace currently used by my process. I don't know if this is available.