How to select and delete columns with duplicate name in pandas DataFrame

Question

I have a huge DataFrame, where some columns have the same names. When I try to pick a column that exists twice, (eg del df[\'col name\'] or df2=

Answer 1

Another solution:

def remove_dup_columns(frame):
     keep_names = set()
     keep_icols = list()
     for icol, name in enumerate(frame.columns):
          if name not in keep_names:
               keep_names.add(name)
               keep_icols.append(icol)
     return frame.iloc[:, keep_icols]

import numpy as np
import pandas as pd

frame = pd.DataFrame(np.random.randint(0, 50, (5, 4)), columns=['A', 'A', 'B', 'B'])

print(frame)
print(remove_dup_columns(frame))

The output is

    A   A   B   B
0  18  44  13  47
1  41  19  35  28
2  49   0  30  16
3  39  29  43  41
4  26  19  48  13
    A   B
0  18  13
1  41  35
2  49  30
3  39  43
4  26  48