StringIO and compatibility with 'with' statement (context manager)

Question

I have some legacy code with a legacy function that takes a filename as an argument and processes the file contents. A working facsimile of the code is below.

What

Answer 1

you could define your own open function

fopen = open
def open(fname,mode):
    if hasattr(fname,"readlines"): return fname
    else: return fopen(fname,mode)

however with wants to call __exit__ after its done and StringIO does not have an exit method...

you could define a custom class to use with this open

class MyStringIO:
     def __init__(self,txt):
         self.text = txt
     def readlines(self):
          return self.text.splitlines()
     def __exit__(self):
          pass

Answer 2

A StringIO instance is an open file already. The open command, on the other hand, only takes filenames, to return an open file. A StringIO instance is not suitable as a filename.

Also, you don't need to close a StringIO instance, so there is no need to use it as a context manager either.

If all your legacy code can take is a filename, then a StringIO instance is not the way to go. Use the tempfile module to generate a temporary filename instead.

Here is an example using a contextmanager to ensure the temp file is cleaned up afterwards:

import os
import tempfile
from contextlib import contextmanager

@contextmanager
def tempinput(data):
    temp = tempfile.NamedTemporaryFile(delete=False)
    temp.write(data)
    temp.close()
    try:
        yield temp.name
    finally:
        os.unlink(temp.name)

with tempinput('Some data.\nSome more data.') as tempfilename:
    processFile(tempfilename)

You can also switch to the newer Python 3 infrastructure offered by the io module (available in Python 2 and 3), where io.BytesIO is the more robust replacement for StringIO.StringIO / cStringIO.StringIO. This object does support being used as a context manager (but still can't be passed to open()).

Answer 3

This one is based on the python doc of contextmanager

It's just wrapping StringIO with simple context, and when exit is called, it will return to the yield point, and properly close the StringIO. This avoids the need of making tempfile, but with large string, this will still eat up the memory, since StringIO buffer that string. It works well on most cases where you know the string data is not going to be long

from contextlib import contextmanager

@contextmanager
def buildStringIO(strData):
    from cStringIO import StringIO
    try:
        fi = StringIO(strData)
        yield fi
    finally:
        fi.close()

Then you can do:

with buildStringIO('foobar') as f:
    print(f.read()) # will print 'foobar'