I want to read a file as fast as possible (40k lines) [Edit : the rest is obsolete].
Edit: Andres Jaan Tack suggested a solution based on one thread per file, and I
I'm thinking like this.
You have 8 cores, so make 8 threads. Let each thread parse one block of the file. So you need to get the device/disk block size. When a block has been parsed by a thread let the thread parse a new one not yet "assigned" to a thread.
Another idea I have would to have 2 threads. A parsing thread and a thread just stepping over the file's disk blocks, ie just by reading the first byte of each block thus forcing the file to be read into memory as fast as possible.
But, this could be made into a contest. Nothing beats doing real live runs! and people will show you! :) find a suitable price!