Parsing datetime strings containing nanoseconds

Question

I have some log files with times in the format HH:MM::SS.nano_seconds (e.g. 01:02:03.123456789). I would like to create a datetime in python so I can neatly do math on the time (e.g. take time differences). strptime works well for microseconds using %f. Do the Python datetime and time modules really not support nanoseconds?

Answer 1

You can see from the source that datetime objects don't support anything more fine than microseconds. As pointed out by Mike Pennington in the comments, this is because actual hardware clocks aren't nearly that precise. Wikipedia says that HPET has frequency "at least 10 MHz," which means one tick per 100 nanoseconds.

If you can live with throwing out the last three digits (which probably aren't too meaningful anyway), you could parse this by just slicing the input string to have only six digits after the decimal point and parsing with %f. Otherwise, it looks like you'll have to implement the subtraction yourself.

Answer 2

You can quite naturally use nanoseconds and even more precise time units (ps, fs, as) with numpy. Numpy has its own Datetimes and Timedeltas implementation, so you can try np.datetime64:

import numpy as np
def str_to_ns(time_str):
     """
     input: time in a format `hh:mm:ss.up_to_9_digits`
     """
     h, m, s = time_str.split(":")
     int_s, ns = s.split(".")
     ns = map(lambda t, unit: np.timedelta64(t, unit),
              [h,m,int_s,ns.ljust(9, '0')],['h','m','s','ns'])
     return sum(ns)

Then you can use this function in a following way:

>>> src = "1:2:34.123456789"
>>> out = str_to_ns(src)
>>> print(out)
3754123456789 nanoseconds
>>> out / np.timedelta64(1,'h')
1.0428120713302778
>>> out / np.timedelta64(1,'m')
62.568724279816664
>>> out / np.timedelta64(1,'s')
3754.123456789

Arithmetic is also possible:

>>> t1, t2 = str_to_ns("1:0:12.12345678"), str_to_ns("1:0:12.12")
>>> t1 - t2
numpy.timedelta64(3456780,'ns')

I agree that it's not that natural, but in this manner you can achieve arbitrary high time precision with just numpy.