Interleaved parallel file read slower than sequential read?

Question

I have implemented a small IO class, which can read from multiple and same files on different disks (e.g two hard disks containing the same file). In sequential case, both d

Answer 1

Maybe http://stxxl.sourceforge.net/ might be of any interest for you, too.

Answer 2

As you said, a sequential read on a disk is much faster than a read-skip-read-skip pattern. Hard disks are capable of high bandwidth when reading sequentially, but the seek time (latency) is expensive.

Instead of storing a copy of the file in each disk, try storing block i of the file on disk i (mod 2). This way you can read from both disks sequentially and recombine the result in memory.

Answer 3

If you are sure that you performing no more than one read per disk (otherwise you will have many disk misses), you still create contention on other parts in the computer - the bus, the raid controller (if exists) and so on.

Answer 4

If you want to do a parallel read, break the read into two sequential reads. Find the halfway point and read the first half from the first file and the second half from the second file.