Quick Read and Merge with Data.Table's Fread and Rbindlist

Question

I am looking for a way to quickly read and merge a bunch of data files using data.table\'s fread and rbindlist functions. I think if fread could take a vector of files names

Answer 1

You could do datatablelist = lapply(list.files("my/data/directory/"), fread) and then rbind the resulting list of data frames.

Although lapply is cleaner than an explicit loop, your loop will work if you read the files directly into a list.

datatablelist = list()

for(i in 1:length(datafiles)){
  datatablelist[[datafiles[i]]] = fread(datafiles[i])
}

Answer 2

Here is a simple way to bind multiple data frames into one single data frame using fread

# Load library
  library(data.table)

# Get a List of all files named with a key word, say all `.csv` files
  filenames <- list.files("C:/your/folder", pattern=glob2rx("*.csv"), full.names=TRUE)

 # Load and bind all data sets
   data <- rbindlist(lapply(filenames,fread))

And in case you want to bind all data files into a list of data frames, it's as simple as

# Load data sets
  list.DFs <- lapply(filenames,fread)