Python: write and read blocks of binary data to a file

我只是一个虾纸丫 提交于 2019-12-06 09:26:49

A file is a stream of bytes without any implied structure. If you want to load a list of binary blobs then you should store some additional metadata to restore the structure e.g., you could use the netstring format:

#!/usr/bin/env python
blocks = [b'\xa1\r\xa594\x92z\xf8\x16\xaa', b'xfbI\xfdqx|\xcd\xdb\x1b\xb3']

# save blocks
with open('blocks.netstring', 'wb') as output_file:
    for blob in blocks:
        # [len]":"[string]","
        output_file.write(str(len(blob)).encode())
        output_file.write(b":")
        output_file.write(blob)
        output_file.write(b",")

Read them back:

#!/usr/bin/env python3
import re
from mmap import ACCESS_READ, mmap

blocks = []
match_size = re.compile(br'(\d+):').match
with open('blocks.netstring', 'rb') as file, \
     mmap(file.fileno(), 0, access=ACCESS_READ) as mm:
    position = 0
    for m in iter(lambda: match_size(mm, position), None):
        i, size = m.end(), int(m.group(1))
        blocks.append(mm[i:i + size])
        position = i + size + 1 # shift to the next netstring
print(blocks)

As an alternative, you could consider BSON format for your data or ascii armor format.

I think what you're looking for is byte_list=open('C:/Python34/testfile.txt','rb').read()

If you know how many bytes each item is, you can use read(number_of_bytes) to process one item at a time.

read() will read the entire file, but then it is up to you to decode that entire list of bytes into their respective items.

In general, since you're using Python 3, you will be working with bytes objects (which are immutable) and/or bytearray objects (which are mutable).

Example:

b1 = bytearray('hello', 'utf-8')
print b1

b1 += bytearray(' goodbye', 'utf-8')
print b1

open('temp.bin', 'wb').write(b1)

#------

b2 = open('temp.bin', 'rb').read()
print b2

Output:

bytearray(b'hello')
bytearray(b'hello goodbye')
b'hello goodbye'
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!