Is it possible to use threads to speed up file reading?

后端 未结 6 1072
情话喂你
情话喂你 2020-12-05 07:47

I want to read a file as fast as possible (40k lines) [Edit : the rest is obsolete].

Edit: Andres Jaan Tack suggested a solution based on one thread per file, and I

6条回答
  •  有刺的猬
    2020-12-05 08:39

    I'm thinking like this.

    You have 8 cores, so make 8 threads. Let each thread parse one block of the file. So you need to get the device/disk block size. When a block has been parsed by a thread let the thread parse a new one not yet "assigned" to a thread.

    Another idea I have would to have 2 threads. A parsing thread and a thread just stepping over the file's disk blocks, ie just by reading the first byte of each block thus forcing the file to be read into memory as fast as possible.

    But, this could be made into a contest. Nothing beats doing real live runs! and people will show you! :) find a suitable price!

提交回复
热议问题