I would like to melt several groups of columns of a dataframe into multiple target columns. Similar to questions Python Pandas Melt Groups of Initial Columns Into Multiple T
You can convert the column names to multi index based on the columns pattern and then stack at a particular level depending on the result you need:
import pandas as pd
df.set_index('id', inplace=True)
df.columns = pd.MultiIndex.from_tuples(tuple(df.columns.str.split("_")))
df.stack(level = 1).reset_index(level = 1, drop = True).reset_index()
# id a b c
#101 a 1 aa
#101 b 2 bb
#101 c 3 cc
#102 d 4 dd
#102 e 5 ee
#102 f 6 ff
There is a more efficient way to do these type of problems that involve melting multiple different sets of columns. pd.wide_to_long
is built for these exact situations.
pd.wide_to_long(df, stubnames=['a', 'b', 'c'], i='id', j='dropme', sep='_')\
.reset_index()\
.drop('dropme', axis=1)\
.sort_values('id')
id a b c
0 101 a 1 aa
2 101 b 2 bb
4 101 c 3 cc
1 102 d 4 dd
3 102 e 5 ee
5 102 f 6 ff
cols = df.columns.difference(['id'])
pd.lreshape(df, cols.groupby(cols.str.split('_').str[0])).sort_values('id')
Out:
id a c b
0 101 a aa 1
2 101 b bb 2
4 101 c cc 3
1 102 d dd 4
3 102 e ee 5
5 102 f ff 6