I have a 14GB data.txt file. I was comparing the speed of fread
and read.table
by reading the first 1M rows. It looks like fread
is mu
fread
mmap
s the file. This takes some time, and will map the whole file. This means subsequent "read-ins" will be faster.
read.table
does not mmap
the whole file. It can read in the file line by line [and stop at line 1000000].
You can see some background on mmap
at mmap() vs. reading blocks
The examples in the help from fread
highlight this behaiviour