Set column names when stacking pandas DataFrame

Question

When stacking a pandas DataFrame, a Series is returned. Normally after I stack a DataFrame, I convert it back into a DataFrame

Answer 1

A pipe-ing friendly alternative to chrisb's answer:

df.stack().rename_axis(['id', 'date', 'var_name']).rename('value').reset_index()

And if explicit is better than implicit:

(
    df
    .stack()
    .rename_axis(index={'id': 'id', 'date': 'date', None: 'var_name'})
    .rename('value')
    .reset_index()
)

When using the dict mapper, you can skip the names which should stay the same:

df.stack().rename_axis(index={None: 'var_name'}).rename('value').reset_index()

Answer 2

So here's one way that you may find a bit cleaner, using the fact that columns and Series can also carry names.

In [45]: df
Out[45]: 
               value  value2
id date                     
1  2015-09-31    100     200
2  2015-09-31     95      57
3  2015-09-31     42      27

In [46]: df.columns.name = 'var_name'

In [47]: s = df.stack()

In [48]: s.name = 'value'

In [49]: s.reset_index()
Out[49]: 
   id        date var_name  value
0   1  2015-09-31    value    100
1   1  2015-09-31   value2    200
2   2  2015-09-31    value     95
3   2  2015-09-31   value2     57
4   3  2015-09-31    value     42
5   3  2015-09-31   value2     27

Answer 3

pd.melt is often useful for converting DataFrames from "wide" to "long" format. You could use pd.melt here if you convert the id and date index levels to columns first:

In [56]: pd.melt(df.reset_index(), id_vars=['id', 'date'], value_vars=['value', 'value2'], var_name='var_name', value_name='value')
Out[56]: 
   id        date var_name  value
0   1  2015-09-31    value    100
1   2  2015-09-31    value     95
2   3  2015-09-31    value     42
3   1  2015-09-31   value2    200
4   2  2015-09-31   value2     57
5   3  2015-09-31   value2     27