Parsing datetime strings containing nanoseconds

问题

I have some log files with times in the format HH:MM::SS.nano_seconds (e.g. 01:02:03.123456789). I would like to create a datetime in python so I can neatly do math on the time (e.g. take time differences). strptime works well for microseconds using %f. Do the Python datetime and time modules really not support nanoseconds?

回答1:

You can see from the source that datetime objects don't support anything more fine than microseconds. As pointed out by Mike Pennington in the comments, this is because actual hardware clocks aren't nearly that precise. Wikipedia says that HPET has frequency "at least 10 MHz," which means one tick per 100 nanoseconds.

If you can live with throwing out the last three digits (which probably aren't too meaningful anyway), you could parse this by just slicing the input string to have only six digits after the decimal point and parsing with %f. Otherwise, it looks like you'll have to implement the subtraction yourself.

回答2:

You can quite naturally use nanoseconds and even more precise time units (ps, fs, as) with numpy. Numpy has its own Datetimes and Timedeltas implementation, so you can try np.datetime64:

import numpy as np
def str_to_ns(time_str):
     """
     input: time in a format `hh:mm:ss.up_to_9_digits`
     """
     h, m, s = time_str.split(":")
     int_s, ns = s.split(".")
     ns = map(lambda t, unit: np.timedelta64(t, unit),
              [h,m,int_s,ns.ljust(9, '0')],['h','m','s','ns'])
     return sum(ns)

Then you can use this function in a following way:

>>> src = "1:2:34.123456789"
>>> out = str_to_ns(src)
>>> print(out)
3754123456789 nanoseconds
>>> out / np.timedelta64(1,'h')
1.0428120713302778
>>> out / np.timedelta64(1,'m')
62.568724279816664
>>> out / np.timedelta64(1,'s')
3754.123456789

Arithmetic is also possible:

>>> t1, t2 = str_to_ns("1:0:12.12345678"), str_to_ns("1:0:12.12")
>>> t1 - t2
numpy.timedelta64(3456780,'ns')

I agree that it's not that natural, but in this manner you can achieve arbitrary high time precision with just numpy.

回答3:

This is an old thread, but still...

You can use Pandas functionality to achieve this. I had timestamps like '2019-03-22T14:00:01.700311864Z' which I converted to a Timestamp by:

    firstStamp = pd.to_datetime(firstStampString, format='%Y-%m-%dT%H:%M:%S.%fZ')
    lastStamp = pd.to_datetime(lastStampString, format='%Y-%m-%dT%H:%M:%S.%fZ')

    deltaTime = lastStamp - firstStamp

This works fine.

回答4:

I found this looking for docs on Firestore timestamps which are returned as DatetimeWithNanoseconds. Turns out it's a custom class of there's which may be of interest here.

To be clear, Firestore is not setting nanoseconds despite the use of this class, but signified by the resolution property of 0.000001 i.e.

来源：https://stackoverflow.com/questions/10611328/parsing-datetime-strings-containing-nanoseconds

标签

python

datetime

strptime

time-precision