Fastest way to import CSV files in MATLAB

Question

I\'ve written a script that saves its output to a CSV file for later reference, but the second script for importing the data takes an ungainly amount of time to read it back

Answer 1

I've had the same problem with reading csv data in Matlab, and I was surprised by how little support there is for this, but then I just found the import data tool. I'm in r2015b.

On the top bar in the "Home" tab, click on "Import Data" and choose the file you'd like to read. An app window will come up like this:

Import Data tool screenshot

Under "Import Selection" you have the option to "generate function", which gives you quite a bit of customization options, including how to fill empty cells, and what you'd like the output data structure to be. Plus it's written by MathWorks, so it's probably utilizing the fastest available method to read csv files. It was almost instantaneous on my file.

Answer 2

It would probably make the data easier to read if you could pad the file with NaN values when your first script creates it:

Item1,1,2,3,NaN
Item2,4,5,6,7
Item3,8,9,NaN,NaN

or you could even just print empty fields:

Item1,1,2,3,
Item2,4,5,6,7
Item3,8,9,,

Of course, in order to pad properly you would need to know what the maximum number of values across all the items is before hand. With either format above, you could then use one of the standard file reading functions, like TEXTSCAN for example:

>> fid = fopen('uneven_data.txt','rt');
>> C = textscan(fid,'%s %f %f %f %f','Delimiter',',','CollectOutput',1);
>> fclose(fid);
>> C{1}

ans = 

    'Item1'
    'Item2'
    'Item3'

>> C{2}

ans =

     1     2     3   NaN  %# TEXTSCAN sets empty fields to NaN anyway
     4     5     6     7
     8     9   NaN   NaN

Answer 3

Instead of parsing the string textline one character at a time. You could use strtok to break the string up for example

stringParts = {};
tline = fgetl(fid);
if ~ischar(tline), break, end
i=1;
while 1
    [stringParts{i},r]=strtok(tline,',');
    tline=r;
    i=i+1;
    if isempty(r), break; end
end

% store the header
headers{count} = stringParts{1};

% convert the data into numbers
for j=2:length(stringParts)
    data{count}(j-1) = str2double(stringParts{j});
end
count=count+1;

Answer 4

Q1) If you know the max number of columns you can fill empty entries with NaN Also, if all values are numerical, do you really need "Item#" column? If yes, you can use only "#", so all data is numerical.

Q2) The fastest way to read num. data from a file without mex-files is csvread. I try to avoid using strings in csv files, but if I have to, I use my csv2cell function:

http://www.mathworks.com/matlabcentral/fileexchange/20135-csv2cell