I tried to concat() two parquet file with pandas in python .
It can work , but when I try to write and save the Data frame to a parquet file ,it display the error :
I experienced a similar problem while using pd.to_parquet
, my final workaround was to use the argument engine='fastparquet'
, but I realize this doesn't help if you need to use PyArrow specifically.
Things I tried which did not work:
.astype("datetime64[ms]")
did not work for me (pandas v. 0.24.2
) coerce_timestamps='ms'
as a kwarg to the underlying parquet operation did not change behaviour.