Reading binary file with python without knowing structure

Question

I have a binary file containing the position of 8000 particles. I know that each particle value should look like "-24.6151..." (I don't know with which precision the values are given by my program. I guess it is double precision(?).

But when I try to read the file with this code:

In: with open('.//results0epsilon/energybinary/energy_00004.dat', 'br') as f:
    buffer = f.read()
    print ("Lenght of buffer is %d" % len(buffer))

    for i in buffer:
        print(int(i))

I get as output:

Lenght of buffer is 64000

10

168

179

43
...

I skip the whole list of values but as you can see those values are far away from what I expect. I think I have some kind of decoding error.

I would appreciate any kind of help :)

Answer 1

What you are printing now are the bytes composing your floating point data. So it doesn't make sense as numerical values.

Of course, there's no 100% sure answer since we didn't see your data, but I'll try to guess:

You have 8000 values to read and the file size is 64000. So you probably have double IEEE values (8 bytes each). If it's not IEEE, then you're toast.

In that case you could try the following:

import struct
with open('.//results0epsilon/energybinary/energy_00004.dat', 'br') as f:
    buffer = f.read()
    print ("Length of buffer is %d" % len(buffer))

    data = struct.unpack("=8000d",buffer)

if the data is printed bogus, it's probably an endianness problem. So change the =8000 by <8000 or >8000.

for reference and packing/unpacking formats: https://docs.python.org/3/library/struct.html