By default, Linux will use free RAM (almost all of it) to cache disk accesses, and will delay writes. The heuristics used by the kernel to decide the caching strategy are not perfect, but beating them in a specific situation is not easy. Also, on journalling filesystems (i.e. all the default filesystems nowadays), actual writes to the disk will be performed in a way which is resilient the crashes; this implies a bit of overhead. You may want to try to fiddle with filesystem options. E.g., for ext3
, try mounting with data=writeback
or even async
(these options may improve filesystem performance, at the expense of reduced resilience towards crashes). Also, use noatime
to reduce filesystem activity.
Programmatically, you might also want to perform disk accesses through memory mappings (with mmap
). This is a bit hand-on, but it gives more control about data management and optimization.