Comparing speed of fread vs. read.table for reading the first 1M rows out of 100M

前端 未结 1 1293
失恋的感觉
失恋的感觉 2020-12-19 08:46

I have a 14GB data.txt file. I was comparing the speed of fread and read.table by reading the first 1M rows. It looks like fread is mu

相关标签:
1条回答
  • 2020-12-19 09:23

    fread mmaps the file. This takes some time, and will map the whole file. This means subsequent "read-ins" will be faster.

    read.table does not mmap the whole file. It can read in the file line by line [and stop at line 1000000].

    You can see some background on mmap at mmap() vs. reading blocks

    The examples in the help from fread highlight this behaiviour

    0 讨论(0)
提交回复
热议问题