How to concatenate all CSVs in a directory, adding CSV name as a column with Python

Question

• I have a folder with about 100 CSVs (Downloads/challenges).
• Each CSV has the same 50+ columns.
• Each CSV is titled something like

Answer 1

This is possible with os from the standard library and 3rd party library pandas:

import os
import pandas as pd

mypath = os.path.join('Downloads', 'challenges')

# get list of files
files = [f for f in os.listdir(mypath) if os.path.isfile(os.path.join(mypath, f))]

# build list of dataframes, adding "challenge" column
dfs = [pd.read_csv(os.path.join(mypath, f)).assign(challenge=f) for f in files]

# concatenate dataframes into one
df = pd.concat(dfs, ignore_index=True)

# write to csv
df.to_csv('all_entrants.csv')