Equivalent of transform in R/ddply in Python/pandas?

Question

In R\'s ddply function, you can compute any new columns group-wise, and append the result to the original dataframe, such as:

ddply(mtcars, .(cyl), transform, n=         


        

Answer 1

You can add a column to a DataFrame by assigning the result of a groupby/transform operation to it:

mtcars['n'] = mtcars.groupby("cyl")['cyl'].transform('count')

---

import pandas as pd
import pandas.rpy.common as com

mtcars = com.load_data('mtcars')
mtcars['n'] = mtcars.groupby("cyl")['cyl'].transform('count')
print(mtcars.head())

yields

                    mpg  cyl  disp   hp  drat     wt   qsec  vs  am  gear  carb   n
Mazda RX4          21.0    6   160  110  3.90  2.620  16.46   0   1     4     4   7
Mazda RX4 Wag      21.0    6   160  110  3.90  2.875  17.02   0   1     4     4   7
Datsun 710         22.8    4   108   93  3.85  2.320  18.61   1   1     4     1  11
Hornet 4 Drive     21.4    6   258  110  3.08  3.215  19.44   1   0     3     1   7
Hornet Sportabout  18.7    8   360  175  3.15  3.440  17.02   0   0     3     2  14

---

To add multiple columns, you could use groupby/apply. Make sure the function you apply returns a DataFrame with the same index as its input. For example,

mtcars[['n','total_wt']] = mtcars.groupby("cyl").apply(
    lambda x: pd.DataFrame({'n': len(x['cyl']), 'total_wt': x['wt'].sum()},
                           index=x.index))
print(mtcars.head())

yields

                    mpg  cyl  disp   hp  drat     wt   qsec  vs  am  gear  carb   n  total_wt
Mazda RX4          21.0    6   160  110  3.90  2.620  16.46   0   1     4     4   7    21.820
Mazda RX4 Wag      21.0    6   160  110  3.90  2.875  17.02   0   1     4     4   7    21.820
Datsun 710         22.8    4   108   93  3.85  2.320  18.61   1   1     4     1  11    25.143
Hornet 4 Drive     21.4    6   258  110  3.08  3.215  19.44   1   0     3     1   7    21.820
Hornet Sportabout  18.7    8   360  175  3.15  3.440  17.02   0   0     3     2  14    55.989