Custom sorting in pandas dataframe

Question

I have python pandas dataframe, in which a column contains month name.

How can I do a custom sort using a dictionary, for example:

custom_dict = {\'         


        

Answer 1

A bit late to the game, but here's a way to create a function that sorts pandas Series, DataFrame, and multiindex DataFrame objects using arbitrary functions.

I make use of the df.iloc[index] method, which references a row in a Series/DataFrame by position (compared to df.loc, which references by value). Using this, we just have to have a function that returns a series of positional arguments:

def sort_pd(key=None,reverse=False,cmp=None):
    def sorter(series):
        series_list = list(series)
        return [series_list.index(i) 
           for i in sorted(series_list,key=key,reverse=reverse,cmp=cmp)]
    return sorter

You can use this to create custom sorting functions. This works on the dataframe used in Andy Hayden's answer:

df = pd.DataFrame([
    [1, 2, 'March'],
    [5, 6, 'Dec'],
    [3, 4, 'April']], 
  columns=['a','b','m'])

custom_dict = {'March':0, 'April':1, 'Dec':3}
sort_by_custom_dict = sort_pd(key=custom_dict.get)

In [6]: df.iloc[sort_by_custom_dict(df['m'])]
Out[6]:
   a  b  m
0  1  2  March
2  3  4  April
1  5  6  Dec

This also works on multiindex DataFrames and Series objects:

months = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']

df = pd.DataFrame([
    ['New York','Mar',12714],
    ['New York','Apr',89238],
    ['Atlanta','Jan',8161],
    ['Atlanta','Sep',5885],
  ],columns=['location','month','sales']).set_index(['location','month'])

sort_by_month = sort_pd(key=months.index)

In [10]: df.iloc[sort_by_month(df.index.get_level_values('month'))]
Out[10]:
                 sales
location  month  
Atlanta   Jan    8161
New York  Mar    12714
          Apr    89238
Atlanta   Sep    5885

sort_by_last_digit = sort_pd(key=lambda x: x%10)

In [12]: pd.Series(list(df['sales'])).iloc[sort_by_last_digit(df['sales'])]
Out[12]:
2    8161
0   12714
3    5885
1   89238

To me this feels clean, but it uses python operations heavily rather than relying on optimized pandas operations. I haven't done any stress testing but I'd imagine this could get slow on very large DataFrames. Not sure how the performance compares to adding, sorting, then deleting a column. Any tips on speeding up the code would be appreciated!