Python Pandas - Concat two data frames with different number of rows and columns

问题

I have two data frames with different row numbers and columns. Both tables has few common columns including "Customer ID". Both tables look like this with a size of 11697 rows × 15 columns and 385839 rows × 6 columns respectively. Customer ID might be repeating in second table. I want to concat both of the tables and want to merge similar columns using Customer ID. How can I do that with python PANDAS. One table looks like this -

and the other one looks like this -

I am using below code -

 pd.concat([df1, df2], sort=False)

Just wanted to make sure that I am not losing any information ? How can I check if there are multiple entries with one ID and how can I combine it in one result ?

EDIT -

When I am using above code, here is before and after values of NA'S in the dataset -

Can someone tell, where I went wrong ?

回答1:

I believe that DataFrame.merge would work in this case:

# use how='outer' to preserve all information from both DataFrames
df1.merge(df2, how='outer', on='customer_id')

DataFrame.join could also work if both DataFrames had their indexes set to customer_id (it is also simpler):

df1 = df1.set_index('customer_id')
df2 = df2.set_index('customer_id')
df1.join(df2, how='outer')

• Documentation for DataFrame.merge
• Documentation for DataFrame.join

回答2:

pd.concat will do the trick here,just set axis to 1 to concatenate on the second axis(columns), you should set the index to customer_id for both data frames first

import pandas as pd
pd.concat([df1.set_index('customer_id'), df2.set_index('customer_id')], axis = 1)

if you want to omit the rows with empty values as a result of your concatenaton, use dropna:

pd.concat([df1.set_index('customer_id'), df2.set_index('customer_id')], axis = 1).dropna()

来源：https://stackoverflow.com/questions/56608366/python-pandas-concat-two-data-frames-with-different-number-of-rows-and-columns