Speed up python's struct.unpack

Question

I am trying to speed up my script. It basically reads a pcap file with Velodyne\'s Lidar HDL-32 information and allows me to get X, Y, Z, and Intensity values. I have profiled m

Answer 1

Compile a Struct ahead of time, to avoid the Python level wrapping code using the module level methods. Do it outside the loops, so the construction cost is not paid repeatedly.

unpack_ushort = struct.Struct('

The Struct methods themselves are implemented in C in CPython (and the module level methods are eventually delegating to the same work after parsing the format string), so building the Struct once and storing bound methods saves a non-trivial amount of work, particularly when unpacking a small number of values.

You can also save some work by unpacking multiple values together, rather than one at a time:

distanceInformation, intensity = unpack_ushort_byte(firingData[startingByte:startingByte + 3])
distanceInformation *= 0.002

As Dan notes, you could further improve this with iter_unpack, which would further reduce the amount of byte code execution and small slice operations.