python 3: reading bytes from stdin pipe with readahead

Question

i want to read bytes. sys.stdin is opened in textmode, yet it has a buffer that can be used to read bytes: sys.stdin.buffer.

my problem is

Answer 1

user4815162342's solution, while extremely useful, appears to have an issue in that it differs from the current behaviour of the io.BufferedReader peek method.

The builtin method will return the same data (starting from the current read position) for sequential peek() calls.

user4815162342's solution will return sequential chunks of data for each sequential peek call. This implies the user must wrap peek again to concatenate the output if they wish to use the same data more than once.

Here is the fix to return builtin behaviour:

def _buffered(self):
    oldpos = self.buf.tell()
    data = self.buf.read()
    self.buf.seek(oldpos)
    return data

def peek(self, size):
    buf = self._buffered()[:size]
    if len(buf) < size:
        contents = self.fileobj.read(size - len(buf))
        self._append_to_buf(contents)
        return self._buffered()
    return buf

See the full version here

There are other optimisations that could be applied, e.g. removal of previously buffered data upon a read call that exhausts the buffer. The current implementation leaves any peeked data in the buffer, but that data is inaccessible.