Pandas DatetimeIndex converting dates to 1970

Question

I have recently faced a similar problem (answered here) whereby conversion of a date to a pandas DatetimeIndex and subsequent groupby using those dates led to an error where the date appeared as 1970-01-01 00:00:00+00:00.

I'm facing this problem in a different context now, and the previous solution isn't helping me.

I have a frame like this

import pandas as pd
from dateutil import tz

data = { 'Events' : range(1, 5 + 1 ,1), 'ID' : [1, 1, 1, 1, 1]}
idx = pd.date_range(start='2008-01-01', end='2008-01-05', freq='D', tz=tz.tzlocal())
frame = pd.DataFrame(data, index=idx)



                           Events  ID
2008-01-01 00:00:00+00:00       1   1
2008-01-02 00:00:00+00:00       2   1
2008-01-03 00:00:00+00:00       3   1
2008-01-04 00:00:00+00:00       4   1
2008-01-05 00:00:00+00:00       5   1

and I want to change the index from just the date, to a MultiIndex of [date, ID], but in doing so that "1970 bug" appears

frame.set_index([frame.ID, frame.index])

                              Events  ID
ID                                      
1  2008-01-01 00:00:00+00:00       1   1
   1970-01-01 00:00:00+00:00       2   1
   1970-01-01 00:00:00+00:00       3   1
   1970-01-01 00:00:00+00:00       4   1
   1970-01-01 00:00:00+00:00       5   1

Versions

• Python 2.7.11
• Pandas 0.18.0

Answer 1

The accepted answer of your other question works for me (Python 3.5.2, Pandas 0.18.1):

print(frame.set_index([frame.ID, frame.index]))

#                               Events  ID
# ID                                      
# 1  2008-01-01 00:00:00-05:00       1   1
#    1970-01-01 00:00:00-05:00       2   1
#    1970-01-01 00:00:00-05:00       3   1
#    1970-01-01 00:00:00-05:00       4   1
#    1970-01-01 00:00:00-05:00       5   1

frame.index = frame.index.tz_convert(tz='EST')
print(frame.set_index([frame.ID, frame.index]))

#                               Events  ID
# ID                                      
# 1  2008-01-01 00:00:00-05:00       1   1
#    2008-01-02 00:00:00-05:00       2   1
#    2008-01-03 00:00:00-05:00       3   1
#    2008-01-04 00:00:00-05:00       4   1
#    2008-01-05 00:00:00-05:00       5   1

(My local time is different from yours.)

Answer 2

frame = frame.reset_index()
frame = frame.set_index([frame.ID, frame.index])
print frame

                         index  Events  ID
ID                                        
1  0 2008-01-01 00:00:00-05:00       1   1
   1 2008-01-02 00:00:00-05:00       2   1
   2 2008-01-03 00:00:00-05:00       3   1
   3 2008-01-04 00:00:00-05:00       4   1
   4 2008-01-05 00:00:00-05:00       5   1


print frame.info()

<class 'pandas.core.frame.DataFrame'>
MultiIndex: 5 entries, (1, 0) to (1, 4)
Data columns (total 4 columns):
level_0    5 non-null int64
index      5 non-null datetime64[ns, tzlocal()]
Events     5 non-null int64
ID         5 non-null int64
dtypes: datetime64[ns, tzlocal()](1), int64(3)
memory usage: 200.0+ bytes