I am reading multiple spreadsheets of timeseries into a pandas dataFrame and concatenating them together with a common pandas datetime index. The datalogger that logged the
For data columns; Use: round_hour(df.Start_time)
def round_hour(x,tt=''):
if tt=='M':
return pd.to_datetime(((x.astype('i8')/(1e9*60)).round()*1e9*60).astype(np.int64))
elif tt=='H':
return pd.to_datetime(((x.astype('i8')/(1e9*60*60)).round()*1e9*60*60).astype(np.int64))
else:
return pd.to_datetime(((x.astype('i8')/(1e9)).round()*1e9).astype(np.int64))
Here's a little trick. Datetimes are in nanoseconds (when viewed as np.int64
).
So round to minutes in nanoseconds.
In [75]: index = pd.DatetimeIndex([ Timestamp('20120827 12:05:00.002'), Timestamp('20130101 12:05:01'), Timestamp('20130712 15:10:00'), Timestamp('20130712 15:10:00.000004') ])
In [79]: index.values
Out[79]:
array(['2012-08-27T08:05:00.002000000-0400',
'2013-01-01T07:05:01.000000000-0500',
'2013-07-12T11:10:00.000000000-0400',
'2013-07-12T11:10:00.000004000-0400'], dtype='datetime64[ns]')
In [78]: pd.DatetimeIndex(((index.asi8/(1e9*60)).round()*1e9*60).astype(np.int64)).values
Out[78]:
array(['2012-08-27T08:05:00.000000000-0400',
'2013-01-01T07:05:00.000000000-0500',
'2013-07-12T11:10:00.000000000-0400',
'2013-07-12T11:10:00.000000000-0400'], dtype='datetime64[ns]')
Issue 4314 mentioned by Jeff is now closed and round()
method was added for DatetimeIndex, Timestamp, TimedeltaIndex and Timedelta in pandas 0.18.0. Now we can do the following:
In[109]: index = pd.DatetimeIndex([pd.Timestamp('20120827 12:05:00.002'), pd.Timestamp('20130101 12:05:01'), pd.Timestamp('20130712 15:10:30'), pd.Timestamp('20130712 15:10:31')])
In[110]: index.values
Out[110]:
array(['2012-08-27T12:05:00.002000000', '2013-01-01T12:05:01.000000000',
'2013-07-12T15:10:30.000000000', '2013-07-12T15:10:31.000000000'], dtype='datetime64[ns]')
In[111]: index.round('min')
Out[111]:
DatetimeIndex(['2012-08-27 12:05:00', '2013-01-01 12:05:00',
'2013-07-12 15:10:00', '2013-07-12 15:11:00'],
dtype='datetime64[ns]', freq=None)
round()
accepts frequency parameter. String aliases for it are listed here.