Return file handles opened with with open?

问题

I'm creating software where I want to accept compressed files. Since files are read/written everywhere, I created a utility function for opening files, that handles opening/closing for me, for some compressed filetypes.

Example code:

def return_file_handle(input_file, open_mode="r"):
    """ Handles compressed and uncompressed files. Accepts open modes r/w/w+ """

    if input_file.endswith(".gz")
        with gzip.open(input_file, open_mode) as gzipped_file_handle:
            return gzipped_file_handle

The problem is, when using this code, the file handle seems to close when the function returns. I it possible to do what I want with with open or do I need to handle closing myself?

Add this to the code above to get a minimal non-working example:

for line in return_file_handle(input_bed, "rb"):
    print line

Create a gzipped textfile with:

echo "hei\nder!" | gzip - > test.gz

Error message:

Traceback (most recent call last):
  File "check_bed_against_blacklist.py", line 26, in <module>
    check_bed_against_blacklist("test.gz", "bla")
  File "check_bed_against_blacklist.py", line 15, in check_bed_against_blacklist
    for line in return_file_handle(input_bed, "r"):
ValueError: I/O operation on closed file.

回答1:

Try it as a generator:

def return_file_handle(input_file, open_mode="r"):
""" Handles compressed and uncompressed files. Accepts open modes r/w/w+ """

    if input_file.endswith(".gz")
        with gzip.open(input_file, open_mode) as gzipped_file_handle:
            yield gzipped_file_handle

When you call it:

for line in return_file_handle("file.gz"):
    print line.read()

回答2:

The best way I can think of is passing a function as parameter which accept opened fd:

def work(fd):
    for line in fd:
        print line

def work_with_file_handle(input_file, func, open_mode="r"):
   if input_file.endswith(".gz")
       with gzip.open(input_file, open_mode) as gzipped_file_handle:
           func(gzipped_file_handle)

work_with_file_handle('xxx.gz', work)

回答3:

Avoid with, if you want to return the file_handle. Because file_handle will be automatically closed when with block is finished executing.

Following code is what you should be using:

import gzip
def return_file_handle(input_file, open_mode="rb"):
    if input_file.endswith(".gz"):
        gzipped_file_handle = gzip.open(input_file, open_mode)
        return gzipped_file_handle

for line in return_file_handle('file.txt.gz', "r"):
    print line

回答4:

The style you are using to open the file automatically closes it at the end of the block. That's the whole point of the with block style of opening files.

What you want to do is:

gzipped_file_handle = gzip.open(input_file, open_mode)
return gzipped_file_handle

NOTICE: You'll just have to be careful to remember to close the file after your call to this function.

回答5:

I would use another context manager

from contextlib import contextmanager

@contextmanager
def return_file_handle(input_file, open_mode="r"):
    """ Handles compressed and uncompressed files. Accepts open modes r/w/w+ """

    if input_file.endswith(".gz")
        with gzip.open(input_file, open_mode) as gzipped_file_handle:
            yield gzipped_file_handle
    else:
        with open(input_file, open_mode) as normal_file:
            yield normal_file
    # Your file will be closed after this


with return_file_handle(file_name, mode) as f:
     pass

来源：https://stackoverflow.com/questions/28825251/return-file-handles-opened-with-with-open

标签

python

with-statement