pandas: pivot table inside multilevel dataframe

问题

I'm trying to pivot table, in order to transform some rows values in columns, so from this dataframe df_behave

list 
                   date_time      field      value
 1    0 2015-05-22 05:37:59      StudentID   129
      1 2015-05-22 05:37:59      SchoolId    3
      2 2015-05-22 05:37:59      GroupeId     45
 2    3 2015-05-26 05:56:59      StudentID   129
      4 2015-05-26 05:56:59      SchoolId     65
      5 2015-05-26 05:56:59      GroupeId    13
      6 2015-05-26 05:56:59      Reference     87
 3    ......................    ......  ......

in order to achieve :

list 
                      date_time     StudentID   SchoolId  GroupId    Reference
     1       2015-05-22 05:37:59      129           3         45

     2      2015-05-26 05:56:59      129            65        15       87   

     3    ......................    ......  ......

with following code:

def calculate():
    df_behave['value'] = df_behave['value'].astype(int)
    pi_df=pd.pivot_table(df_behave, 'value', index=['date_time'], columns='field')
    return pi_df

and I tried this one:

def calculate():
    df_behave['value'] = df_behave['value'].astype(int)
    for liste, new_df in df_behave.groupby(level=0):
        pi_df=pd.pivot_table(new_df, 'value', index=['date_time'], columns='field')
        print pi_df
    return pi_df

but the both returned me ValueError: invalid literal for long() with base 10: 'True'

回答1:

Try resetting your index, setting it to list, date_time and field, and then unstacking field.

df.reset_index().set_index(['list', 'date_time', 'field']).unstack('field')

As your value column appears to contain non-numeric data, and from your examples above it should only contain integers, try the following to locate your bad data:

bad_rows = []
for n in range(len(df) - 1):
    if not isinstance(df.loc[n, 'value'], int):
        bad_rows.append(n)

You may first want to try coercing the values:

df['value'] = df['value'].astype('int')

回答2:

@Alexander is right, for MultiIndex, you'd better reset_index and set for the fields he mentioned and perform an unstack. Perhaps you should filter out the unnecessary fields?

just some random sample data:

In [308]: df
Out[308]: 
                     date_time     field  value
list index                                     
1    0     2015-05-22 05:37:59       Tom      1
     1     2015-05-22 05:37:59      Kate      2
     2     2015-05-22 05:37:59  GroupeId      3
2    3     2015-05-22 05:37:59       Tom      4
     4     2015-05-22 05:37:59      Kate      5
     5     2015-05-22 05:37:59  GroupeId      6

In [310]: df.set_index(['date_time', 'field'], append=True)\
            .reset_index('index')['value']\
            .unstack('field')
Out[310]: 
field                     GroupeId  Kate  Tom
list date_time                               
1    2015-05-22 05:37:59         3     2    1
2    2015-05-22 05:37:59         6     5    4

来源：https://stackoverflow.com/questions/33174178/pandas-pivot-table-inside-multilevel-dataframe

标签

python

pandas

pivot

dataframe

multi-index