What is the fastest way to read 10 GB file from the disk?

后端 未结 13 3678
你的背包
你的背包 2021-02-20 01:19

We need to read and count different types of messages/run some statistics on a 10 GB text file, e.g a FIX engine log. We use Linux, 32-bit, 4 CPUs, Intel, coding in Perl but the

13条回答
  •  忘掉有多难
    2021-02-20 02:16

    I wish I knew more about the content of your file, but not knowing other than that it is text, this sounds like an excellent MapReduce kind of problem.

    PS, the fastest read of any file is a linear read. cat file > /dev/null should be the speed that the file can be read.

提交回复
热议问题