Calculate difference between 'times' rows in DataFrame Pandas

问题

My DataFrame is in the Form:

       TimeWeek   TimeSat  TimeHoli
0      6:40:00   8:00:00   8:00:00
1      6:45:00   8:05:00   8:05:00
2      6:50:00   8:09:00   8:10:00
3      6:55:00   8:11:00   8:14:00
4      6:58:00   8:13:00   8:17:00
5      7:40:00   8:15:00   8:21:00

I need to find the time difference between each row in TimeWeek , TimeSat and TimeHoli, the output must be

TimeWeekDiff   TimeSatDiff  TimeHoliDiff
00:05:00          00:05:00       00:05:00
00:05:00          00:04:00       00:05:00
00:05:00          00:02:00       00:04:00  
00:03:00          00:02:00       00:03:00
00:02:00          00:02:00       00:04:00

I tried using (d['TimeWeek']-df['TimeWeek'].shift().fillna(0) , it throws an error:

TypeError: unsupported operand type(s) for -: 'str' and 'str'

Probably because of the presence of ':' in the column. How do I resolve this?

回答1:

It looks like the error is thrown because the data is in the form of a string instead of a timestamp. First convert them to timestamps:

df2 = df.apply(lambda x: [pd.Timestamp(ts) for ts in x])

They will contain today's date by default, but this shouldn't matter once you difference the time (hopefully you don't have to worry about differencing 23:55 and 00:05 across dates).

Once converted, simply difference the DataFrame:

>>> df2 - df2.shift()
   TimeWeek  TimeSat  TimeHoli
0       NaT      NaT       NaT
1  00:05:00 00:05:00  00:05:00
2  00:05:00 00:04:00  00:05:00
3  00:05:00 00:02:00  00:04:00
4  00:03:00 00:02:00  00:03:00
5  00:42:00 00:02:00  00:04:00

Depending on your needs, you can just take rows 1+ (ignoring the NaTs):

(df2 - df2.shift()).iloc[1:, :]

or you can fill the NaTs with zeros:

(df2 - df2.shift()).fillna(0)

回答2:

Forget everything I just said. Pandas has great timedelta parsing.

df["TimeWeek"] = pd.to_timedelta(df["TimeWeek"])
(d['TimeWeek']-df['TimeWeek'].shift().fillna(pd.to_timedelta("00:00:00"))

回答3:

>>> import pandas as pd
>>> df = pd.DataFrame({'TimeWeek': ['6:40:00', '6:45:00', '6:50:00', '6:55:00', '7:40:00']})
>>> df["TimeWeek_date"] = pd.to_datetime(df["TimeWeek"], format="%H:%M:%S")
>>> print df
  TimeWeek       TimeWeek_date
0  6:40:00 1900-01-01 06:40:00
1  6:45:00 1900-01-01 06:45:00
2  6:50:00 1900-01-01 06:50:00
3  6:55:00 1900-01-01 06:55:00
4  7:40:00 1900-01-01 07:40:00
>>> df['TimeWeekDiff'] = (df['TimeWeek_date'] - df['TimeWeek_date'].shift().fillna(pd.to_datetime("00:00:00", format="%H:%M:%S")))
>>> print df
  TimeWeek       TimeWeek_date  TimeWeekDiff
0  6:40:00 1900-01-01 06:40:00      06:40:00
1  6:45:00 1900-01-01 06:45:00      00:05:00
2  6:50:00 1900-01-01 06:50:00      00:05:00
3  6:55:00 1900-01-01 06:55:00      00:05:00
4  7:40:00 1900-01-01 07:40:00      00:45:00

来源：https://stackoverflow.com/questions/29573441/calculate-difference-between-times-rows-in-dataframe-pandas

标签

python

pandas

row

difference