ValueError: Cannot cast DatetimeIndex to dtype datetime64[us]

Question

I\'m trying to create a PostgreSQL table of 30-minute data for the S&P 500 ETF (spy30new, for testing freshly inserted data) from a table of several stocks with 15-minute da

Answer 1

I had the same problem and applying pd.to_datetime() on each element worked as well. But it is orders of magnitude slower than running pd.to_datetime() on the entire series. For a dataframe with over a 1 million rows:

(df['Time']).apply(lambda d: pd.to_datetime(str(d)))

takes approximately 70 seconds

and

pd.to_datetime(df['Time'])

takes approximately 0.01 seconds

The actual problem is that timezone information is being included. To remove it:

t = pd.to_datetime(df['Time'])
t = t.tz_localize(None)

This should be much faster!