How to stop page cache for disk I/O in my linux system?

房东的猫 提交于 2019-12-08 06:19:40

问题


Here is my system based on Linux2.6.32.12:
1 It contains 20 processes which occupy a lot of usr cpu 2 It needs to write data on rate 100M/s to disk and those data would not be used recently.

What I expect:
It can run steadily and disk I/O would not affect my system.

My problem:
At the beginning, the system run as I thought. But as the time passed, Linux would cache a lot data for the disk I/O, that lead to physical memory reducing. At last, there will be not enough memory, then Linux will swap in/out my processes. It will cause I/O problem that a lot cpu time was used to I/O.

What I have try:
I try to solved the problem, by "fsync" everytime I write a large block.But the physical memory is still decreasing while cached increasing.

How to stop page cache here, it's useless for me

More infomation:
When Top show free 46963m, all is well including cpu %wa is low and vmstat shows no si or so.
When Top show free 273m, %wa is so high which affect my processes and vmstat shows a lot si and so.


回答1:


I'm not sure that changing something will affect overall performance.

Maybe you might use posix_fadvise(2) and sync_file_range(2) in your program (and more rarely fsync(2) or fdatasync(2) or sync(2) or syncfs(2), ...). Also look at madvise(2), mlock(2) and munlock(2), and of course mmap(2) and munmap(2). Perhaps ionice(1) could help.

In the reader process, you might perhaps use readhahead(2) (perhaps in a separate thread).

Upgrading your kernel (to a 3.6 or better) could certainly help: Linux has improved significantly on these points since 2.6.32 which is really old.




回答2:


To drop pagecache you can do the following:

         "echo 1 > /proc/sys/vm/drop_caches"

drop_caches are usually 0. And, can be changed as per need. As you've identified yourself, that you need to free pagecache, so this is how to do it. You can also take a look at dirty_writeback_centisecs (and it's related tunables)(http://lxr.linux.no/linux+*/Documentation/sysctl/vm.txt#L129) to make quick writeback, but note it might have consequences, as it calls up kernel flasher thread to write out dirty pages. Also, note the uses of dirty_expire_centices, which defines how much time some data needs to be eligible for writeout.



来源:https://stackoverflow.com/questions/16518433/how-to-stop-page-cache-for-disk-i-o-in-my-linux-system

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!