Crafted hex string correct in string format, malforms once passed to unhexlify()

Question

def craft_integration(xintegration_time):

 integration_time = xintegration_time
 integration_time_str = str(integration_time)
 integration_time_str = integration_time_s         


        

Answer 1

What you have in fact is simply your desired value being rendered into a form by the default implementation of bytes.__repr__ that you were not expecting to the point that it was unhelpful to what you want.

To start from a more basic level: in Python, any element (well, any "byte", i.e. a group of 8 bits) inside a bytes type are typically being stored as raw digital representation somewhere in a machine as binary. In order to "print" them out onto a console for human consumption it must be turned into a form that may be interpreted by the console such that the correct glyph may be used to represent the underlying value. For many values, such as 0 (or 00000000 in binary), Python would use \x00 to represent that. The \ is the escape character to start an escape sequence, the x that follows signifies that the escape sequence is to be followed by 2 hexadecimal characters, and combining those two characters with the whole sequence would form the representation of that single byte using four characters. Likewise for 255, in binary that would be 11111111, and this same value as part of a bytes type will be encoded as \xff.

Now there are exceptions - if a given value falls inside the ASCII range, and that it in the range of printable characters, the representation will instead be the corresponding ASCII character. So in the case of the hexadecimal 30 (decimal 48), rendering of that as part of a bytes type will show 0 instead of \x30, as 0 is the corresponding printable character.

So for your case, a bytes representation that was printed out in the console in the form of b'\x041000', is not in fact a big \x value, as the \x escape sequence is only applied to exactly two subsequent characters - all following characters (i.e. 1000) are in fact being represented using the printable characters that would otherwise be represented as \x31\x30\x30\x30.

There is another method available to those who don't mind working with the decimal representation of bytes - simply cast the bytes into a bytearray then into a list. We will take two nul bytes (b'\x00\x00') as an example:

>>> list(bytearray(b'\x00\x00'))
[0, 0]

Clearly those two nul bytes will correspond to two zero values. Now try using the confusing b'\x04\x31\x30\x30\x30' which got rendered into b'\x041000':

>>> list(bytearray(b'\x041000'))
[4, 49, 48, 48, 48]

We can note that it was in fact 5 bytes rendered with the corresponding decimal numbers in a list of 5 elements.

It is often easy to get confused with what the actual value is, vs. what is being shown and visualized on the computer console. Unfortunately the tools we use sometimes amplify that confusion, but as programmers we should understand this and seek ways to minimize this for users of our work, as this example shows that not everyone may have the intuition that certain representations of bytes may instead be represented as printable ASCII.