yielding items from NUL separated input in python

Deadly 提交于 2020-01-04 05:39:15

问题


I have a situation in python where I need to loop items from a NUL separated stream given in a format similar to the output of find /somewhere -print0

  • The stream is binary, items can consist of all bytes except NUL
  • There is no way of knowing if the whole thing would fit in the available memory (assume the stream can be infinite).
  • There is no way of knowing the length of the individual items (assume it could always be longer than READ_SIZE in my current solution below).

I feel like I'm missing something here, like fd.readlines() but for \0 instead of \n)

Here's what I currently use to tackle this:

READ_SIZE = 2**14
def readitems(fd):
    buffer = b''
    last = fd.read(READ_SIZE)
    while(last):
        x = last.split(b'\0')
        for i in range(len(x)-1):
            yield buffer + x[i]
            buffer = b''
        buffer += x[-1]
        last = fd.read(READ_SIZE)

If there's really no built-in method that I'm missing to do this, all faster and/or cleaner solutions are welcome.

来源:https://stackoverflow.com/questions/45482167/yielding-items-from-nul-separated-input-in-python

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!