How can I safely create a nested directory?

Question

What is the most elegant way to check if the directory a file is going to be written to exists, and if not, create the directory using Python? Here is what I tried:

Answer 1

Insights on the specifics of this situation

You give a particular file at a certain path and you pull the directory from the file path. Then after making sure you have the directory, you attempt to open a file for reading. To comment on this code:

filename = "/my/directory/filename.txt"
dir = os.path.dirname(filename)

We want to avoid overwriting the builtin function, dir. Also, filepath or perhaps fullfilepath is probably a better semantic name than filename so this would be better written:

import os
filepath = '/my/directory/filename.txt'
directory = os.path.dirname(filepath)

Your end goal is to open this file, you initially state, for writing, but you're essentially approaching this goal (based on your code) like this, which opens the file for reading:

if not os.path.exists(directory):
    os.makedirs(directory)
f = file(filename)

Assuming opening for reading

Why would you make a directory for a file that you expect to be there and be able to read?

Just attempt to open the file.

with open(filepath) as my_file:
    do_stuff(my_file)

If the directory or file isn't there, you'll get an IOError with an associated error number: errno.ENOENT will point to the correct error number regardless of your platform. You can catch it if you want, for example:

import errno
try:
    with open(filepath) as my_file:
        do_stuff(my_file)
except IOError as error:
    if error.errno == errno.ENOENT:
        print 'ignoring error because directory or file is not there'
    else:
        raise

Assuming we're opening for writing

This is probably what you're wanting.

In this case, we probably aren't facing any race conditions. So just do as you were, but note that for writing, you need to open with the w mode (or a to append). It's also a Python best practice to use the context manager for opening files.

import os
if not os.path.exists(directory):
    os.makedirs(directory)
with open(filepath, 'w') as my_file:
    do_stuff(my_file)

However, say we have several Python processes that attempt to put all their data into the same directory. Then we may have contention over creation of the directory. In that case it's best to wrap the makedirs call in a try-except block.

import os
import errno
if not os.path.exists(directory):
    try:
        os.makedirs(directory)
    except OSError as error:
        if error.errno != errno.EEXIST:
            raise
with open(filepath, 'w') as my_file:
    do_stuff(my_file)