How to limit CPU and RAM resources for mongodump?

我与影子孤独终老i 提交于 2019-12-11 02:02:06

问题


I have a mongod server running. Each day, I am executing the mongodump in order to have a backup. The problem is that the mongodump will take a lot of resources and it will slow down the server (that by the way already runs some other heavy tasks).

My goal is to somehow limit the mongodump which is called in a shell script.

Thanks.


回答1:


You should use cgroups. Mount points and details are different on distros and a kernels. I.e. Debian 7.0 with stock kernel doesn't mount cgroupfs by default and have memory subsystem disabled (folks advise to reboot with cgroup_enabled=memory) while openSUSE 13.1 shipped with all that out of box (due to systemd mostly).

So first of all, create mount points and mount cgroupfs if not yet done by your distro:

mkdir /sys/fs/cgroup/cpu
mount -t cgroup -o cpuacct,cpu cgroup /sys/fs/cgroup/cpu

mkdir /sys/fs/cgroup/memory
mount -t cgroup -o memory cgroup /sys/fs/cgroup/memory

Create a cgroup:

mkdir /sys/fs/cgroup/cpu/shell
mkdir /sys/fs/cgroup/memory/shell

Set up a cgroup. I decided to alter cpu shares. Default value for it is 1024, so setting it to 128 will limit cgroup to 11% of all CPU resources, if there are competitors. If there are still free cpu resources they would be given to mongodump. You may also use cpuset to limit numver of cores available to it.

echo 128 > /sys/fs/cgroup/cpu/shell/cpu.shares
echo 50331648 > /sys/fs/cgroup/memory/shell/memory.limit_in_bytes

Now add PIDs to the cgroup it will also affect all their children.

echo 13065 >  /sys/fs/cgroup/cpu/shell/tasks
echo 13065 >  /sys/fs/cgroup/memory/shell/tasks

I run couple of tests. Python that tries to allocate bunch of mem was Killed by OOM:

myaut@zenbook:~$ python -c 'l = range(3000000)'
Killed

I've also run four infinite loops and fifth in cgroup. As expected, loop that was run in cgroup got only about 45% of CPU time, while the rest of them got 355% (I have 4 cores).

All that changes do not survive reboot!

You may add this code to a script that runs mongodump, or use some permanent solution.



来源:https://stackoverflow.com/questions/28168134/how-to-limit-cpu-and-ram-resources-for-mongodump

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!