问题
I have a dataframe called 'running_tally'
list jan_to jan_from
0 LA True False
1 NY False True
I am trying to append new data to it in the form of a single column dataframe called 'new_data'
list
0 HOU
1 LA
I concat these two dfs based on their 'list' column for further processing, but immediately after I do that all the boolean values unexpectedly flip.
running_tally = pd.concat([running_tally,new_data]).groupby('list',as_index=False).first()
the above statement will produce:
list jan_to jan_from
0 LA False True
1 NY True False
2 HOU NaN NaN
NaN values are expected for the new row, but I don't know why the bools all flip. What could be the reason for this? The code logically makes sense to me so I'm not sure where I'm going wrong. Thanks
EDIT: I made an edit to 'new_data' to include a repeat with LA. The final output should not have repeats which my code currently handles correctly, just has boolean flipping
EDIT 2: Turns out that when concatenating, the columns would flip in order leading me to believe the bools flipped. Still an open issue however
回答1:
I am not sure why you want to use a groupby in this case... when using concat there is no need to specify which columns you want to use, as long as their names are identical. Simple concatenation like this should do:
running_tally = pd.concat([running_tally,new_data], ignore_index=True, sort=False)
EDIT to take question edit into account: this should do the same job, without duplicates.
running_tally = running_tally.merge(new_data, on="list", how="outer")
回答2:
I don´t get the booleans flipped as you, but you can try this too:
running_tally=running_tally.append(new_data,ignore_index=True)
print(running_tally)
Output:
list jan_to jan_from
0 LA True False
1 NY False True
2 HOU NaN NaN
EDIT: Since the question was edited, you could try with:
running_tally=running_tally.append(new_data,ignore_index=True).groupby('list',as_index=False).first()
回答3:
The actual row order was being flipped when using concat for pandas 0.20.1
How to concat pandas Dataframes without changing the column order in Pandas 0.20.1?
来源:https://stackoverflow.com/questions/62665453/pandas-concat-flips-all-my-values-in-the-dataframe