How can I safely create a nested directory?

Question

What is the most elegant way to check if the directory a file is going to be written to exists, and if not, create the directory using Python? Here is what I tried:

Answer 1

Python 3.5+:

import pathlib
pathlib.Path('/my/directory').mkdir(parents=True, exist_ok=True) 

pathlib.Path.mkdir as used above recursively creates the directory and does not raise an exception if the directory already exists. If you don't need or want the parents to be created, skip the parents argument.

Python 3.2+:

Using pathlib:

If you can, install the current pathlib backport named pathlib2. Do not install the older unmaintained backport named pathlib. Next, refer to the Python 3.5+ section above and use it the same.

If using Python 3.4, even though it comes with pathlib, it is missing the useful exist_ok option. The backport is intended to offer a newer and superior implementation of mkdir which includes this missing option.

Using os:

import os
os.makedirs(path, exist_ok=True)

os.makedirs as used above recursively creates the directory and does not raise an exception if the directory already exists. It has the optional exist_ok argument only if using Python 3.2+, with a default value of False. This argument does not exist in Python 2.x up to 2.7. As such, there is no need for manual exception handling as with Python 2.7.

Python 2.7+:

Using pathlib:

If you can, install the current pathlib backport named pathlib2. Do not install the older unmaintained backport named pathlib. Next, refer to the Python 3.5+ section above and use it the same.

Using os:

import os
try: 
    os.makedirs(path)
except OSError:
    if not os.path.isdir(path):
        raise

While a naive solution may first use os.path.isdir followed by os.makedirs, the solution above reverses the order of the two operations. In doing so, it prevents a common race condition having to do with a duplicated attempt at creating the directory, and also disambiguates files from directories.

Note that capturing the exception and using errno is of limited usefulness because OSError: [Errno 17] File exists, i.e. errno.EEXIST, is raised for both files and directories. It is more reliable simply to check if the directory exists.

Alternative:

mkpath creates the nested directory, and does nothing if the directory already exists. This works in both Python 2 and 3.

import distutils.dir_util
distutils.dir_util.mkpath(path)

Per Bug 10948, a severe limitation of this alternative is that it works only once per python process for a given path. In other words, if you use it to create a directory, then delete the directory from inside or outside Python, then use mkpath again to recreate the same directory, mkpath will simply silently use its invalid cached info of having previously created the directory, and will not actually make the directory again. In contrast, os.makedirs doesn't rely on any such cache. This limitation may be okay for some applications.

---

With regard to the directory's mode, please refer to the documentation if you care about it.

Answer 2

Insights on the specifics of this situation

You give a particular file at a certain path and you pull the directory from the file path. Then after making sure you have the directory, you attempt to open a file for reading. To comment on this code:

filename = "/my/directory/filename.txt"
dir = os.path.dirname(filename)

We want to avoid overwriting the builtin function, dir. Also, filepath or perhaps fullfilepath is probably a better semantic name than filename so this would be better written:

import os
filepath = '/my/directory/filename.txt'
directory = os.path.dirname(filepath)

Your end goal is to open this file, you initially state, for writing, but you're essentially approaching this goal (based on your code) like this, which opens the file for reading:

if not os.path.exists(directory):
    os.makedirs(directory)
f = file(filename)

Assuming opening for reading

Why would you make a directory for a file that you expect to be there and be able to read?

Just attempt to open the file.

with open(filepath) as my_file:
    do_stuff(my_file)

If the directory or file isn't there, you'll get an IOError with an associated error number: errno.ENOENT will point to the correct error number regardless of your platform. You can catch it if you want, for example:

import errno
try:
    with open(filepath) as my_file:
        do_stuff(my_file)
except IOError as error:
    if error.errno == errno.ENOENT:
        print 'ignoring error because directory or file is not there'
    else:
        raise

Assuming we're opening for writing

This is probably what you're wanting.

In this case, we probably aren't facing any race conditions. So just do as you were, but note that for writing, you need to open with the w mode (or a to append). It's also a Python best practice to use the context manager for opening files.

import os
if not os.path.exists(directory):
    os.makedirs(directory)
with open(filepath, 'w') as my_file:
    do_stuff(my_file)

However, say we have several Python processes that attempt to put all their data into the same directory. Then we may have contention over creation of the directory. In that case it's best to wrap the makedirs call in a try-except block.

import os
import errno
if not os.path.exists(directory):
    try:
        os.makedirs(directory)
    except OSError as error:
        if error.errno != errno.EEXIST:
            raise
with open(filepath, 'w') as my_file:
    do_stuff(my_file)

Answer 3

Check os.makedirs: (It makes sure the complete path exists.)
To handle the fact the directory might exist, catch OSError. (If exist_ok is False (the default), an OSError is raised if the target directory already exists.)

import os
try:
    os.makedirs('./path/to/somewhere')
except OSError:
    pass

Answer 4

Use this command check and create dir

 if not os.path.isdir(test_img_dir):
     os.mkdir(test_img_dir)

Answer 5

I saw Heikki Toivonen and A-B-B's answers and thought of this variation.

import os
import errno

def make_sure_path_exists(path):
    try:
        os.makedirs(path)
    except OSError as exception:
        if exception.errno != errno.EEXIST or not os.path.isdir(path):
            raise

Answer 6

You can use os.listdir for this:

import os
if 'dirName' in os.listdir('parentFolderPath')
    print('Directory Exists')