How to merge pandas value_counts() to dataframe or use it to subset a dataframe

Question

I used pandas df.value_counts() to find the number of occurrences of particular brands. I want to merge those value counts with the respective brands in the initial dataframe.

Answer 1

df = ...
key_col = "brand"
count_col = "brand_count"

result = (
    df.join(
        df[key_col].value_counts().rename(count_col), 
        how="left", 
        on=key_col)
)

If you need to join the counts to a different dataframe remember to fill NaNs with zeros:

df = ...
other = ...
key_col = "brand"
count_col = "brand_count"

result = (
    other.join(
        df[key_col].value_counts().rename(count_col), 
        how="left", 
        on=key_col)
    .fillna({count_col: 0})
)

Answer 2

i think the best way is to use map

df['brand_count']= df.brand.map(df.brand.value_counts())

this is so much faster than groupby method for example (factor 500 on a 15000 row df) and take only one line

Answer 3

You want to use transform.

import numpy as np
import pandas as pd

np.random.seed(0)

# Create dummy data.
df = pd.DataFrame({'brands': ['brand{0}'.format(n) 
                   for n in np.random.random_integers(0, 5, 10)]})

df['brand_count'] = \
    df.groupby('brands', as_index=False)['brands'].transform(lambda s: s.count())

>>> df
   brands brand_count
0  brand4           1
1  brand5           2
2  brand0           1
3  brand3           4
4  brand3           4
5  brand3           4
6  brand1           1
7  brand3           4
8  brand5           2
9  brand2           1

For reference:

>>> df.brands.value_counts()
brand3    4
brand5    2
brand4    1
brand0    1
brand1    1
brand2    1
Name: brands, dtype: int64

Answer 4

is that what you want:

import numpy as np
import pandas as pd

# generating random DataFrame
brands_list = ['brand{}'.format(i) for i in range(10)]
a = pd.DataFrame({'brands': np.random.choice(brands_list, 100)})
b = pd.DataFrame(np.random.randint(0,10,size=(100, 3)), columns=list('ABC'))
df = pd.concat([a, b], axis=1)
print(df.head())

# generate 'brands' DF
brands = pd.DataFrame(df.brands.value_counts().reset_index())
brands.columns = ['brands', 'count']
print(brands)

# merge 'df' & 'brands_count'
merged = pd.merge(df, brands, on='brands')
print(merged)

PS first big part is just a dataframe generation.

The part which is interesting for you starts with the # generate 'brands' DF comment

Answer 5

Pandas DataFrame's merge and value_counts attributes are pretty fast, so I would combine the two.

df.merge(df['brand'].value_counts().to_frame(), how='left', left_on='brand',
         right_index=True, suffixes=('', 'x'))\
  .rename(columns={'brandx':'brand_count'})