Where to put a configuration file in Python?

Question

In development mode, I have the following directory tree :

| my_project/
| setup.py
| my_project/
    | __init__.py
    | main.py
    | conf/
        | myproject.conf

I use ConfigParser to parse the myproject.conf file.

In my code, it's easy to load the file with a good path : my_project/conf/myproject.conf

The problem is : When I install my project using the setup.py, the configuration file are situated (thanks to setup.py) in the /etc/my_project/myproject.conf and my application in the /usr/lib/python<version>/site-packages/my_project/.

How can I refer my my_project/conf/myproject.conf file in my project in "production" mode, and refer to the local file (my_project/conf/myproject.conf) in "devel" mode.

In addition, I would like to be portable if possible (Work on windows for example).

What's the good practice to do that ?

Answer 1

Have you seen how configuration files work? Read up on "rc" files, as they're sometimes called. "bashrc", "vimrc", etc.

There's usually a multi-step search for the configuration file.

1. Local directory. ./myproject.conf.

2. User's home directory (~user/myproject.conf)

3. A standard system-wide directory (/etc/myproject/myproject.conf)

4. A place named by an environment variable (MYPROJECT_CONF)

The Python installation would be the last place to look.

config= None
for loc in os.curdir, os.path.expanduser("~"), "/etc/myproject", os.environ.get("MYPROJECT_CONF"):
    try: 
        with open(os.path.join(loc,"myproject.conf")) as source:
            config.readfp( source )
    except IOError:
        pass

Answer 2

The appdirs package does a nice job on finding the standard place for installed apps on various platforms. I wonder if extending it to discover or allow some sort of "uninstalled" status for developers would make sense.

Answer 3

If you're using setuptools, see the chapter on using non-package data files. Don't try to look for the files yourself.

Answer 4

Another option is to keep all the .cfg and .ini files in the home directory, like 'boto' does.

import os.path
config_file = os.path.join(os.path.expanduser("~"), '.myproject')

Answer 5

I don't think there is a clean way to deal with that. You could simply choose to test for the existence of the 'local' file, which would work in dev mode. Otherwise, fall back to the production path:

import os.path

main_base = os.path.dirname(__file__)
config_file = os.path.join(main_base, "conf", "myproject.conf")

if not os.path.exists(config_file):
    config_file = PROD_CONFIG_FILE   # this could also be different based on the OS