How can I open multiple files (number of files unknown beforehand) using “with open” statement?

Question

I specifically need to use with open statement for opening the files, because I need to open a few hundred files together and merge them using K-way merge. I un

Answer 1

with open(...) as f: 
    # do stuff 

translates roughly to

f = open(...)
# do stuff
f.close()

In your case, I wouldn't use the with open syntax. If you have a list of filenames, then do something like this

filenames = os.listdir(file_directory)
open_files = map(open, filenames)
# do stuff
for f in open_files:
    f.close()

If you really want to use the with open syntax, you can make your own context manager that accepts a list of filenames

class MultipleFileManager(object):
    def __init__(self, files):
        self.files = files

    def __enter__(self):
        self.open_files = map(open, self.files)
        return self.open_files

    def __exit__(self):
        for f in self.open_files:
            f.close()

And then use it like this:

filenames = os.listdir(file_directory)
with MulitpleFileManager(filenames) as files:
    for f in files:
        # do stuff

The only advantage I see to using a context manager in this case is that you can't forget to close the files. But there is nothing wrong with manually closing the files. And remember, the os will reclaim its resources when your program exits anyway.