My python code works correctly in the below example. My code combines a directory of CSV files and matches the headers. However, I want to take it a step further - how do I
Mike's answer above works perfectly. In case any googlers run into the following error:
>>> TypeError: cannot concatenate object of type "<type 'str'>";
only pd.Series, pd.DataFrame, and pd.Panel (deprecated) objs are valid
It's possibly because the separator is not correct. I was using a custom csv file so the separator was ^
. Becuase of that I needed to include the separator in the pd.read_csv
call.
import os
for csv in globbed_files:
frame = pd.read_csv(csv, sep='^')
frame['filename'] = os.path.basename(csv)
data.append(frame)
This should work:
import os
for csv in globbed_files:
frame = pd.read_csv(csv)
frame['filename'] = os.path.basename(csv)
data.append(frame)
frame['filename']
creates a new column named filename
and os.path.basename()
turns a path like /a/d/c.txt
into the filename c.txt
.