How should I handle importing third-party libraries within my setup.py script?

Question

I'm developing a Python application and in the process of branching off a release. I've got a PyPI server set up on a company server and I've copied a source distribution of my package onto it.

I checked that the package was being hosted on the server and then tried installing it on my local development machine. I ended up with this output:

$ pip3 install --trusted-host 172.16.1.92 -i http://172.16.1.92:5001/simple/ <my-package>
Collecting <my-package>
  Downloading http://172.16.1.92:5001/packages/<my-package>-0.2.0.zip
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\setup.py", line 9, in <module>
        import appdirs
    ModuleNotFoundError: No module named 'appdirs'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\

The reason is that I'm trying to import a third-party library appdirs in my setup.py, which is necessary for me to compute the data_files argument to setup():

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

import os
from collections import defaultdict

import appdirs
from <my-package>.version import __version__ as <my-package>_version

APP_NAME = '<my-app>'
APP_AUTHOR = '<company>'
SYSTEM_COMPONENT_PLUGIN_DIR = os.path.join(appdirs.user_data_dir(APP_NAME, APP_AUTHOR), 'components')

# ...

setup(
    # ...
    data_files=component_files,
)

However, I don't have appdirs installed on my local dev machine and I don't expect the end users to have it either.

Is it acceptable to rely on third-party libraries like this in setup.py, and if so what is the recommended approach to using them? Is there a way I can ensure appdirs gets installed before it's imported in setup.py, or should I just document that appdirs is a required package to install my package?

Answer 1

I'm ignoring licensing issues in this answer. You definetly need to take these into account before you really do a release.

Is it acceptable to rely on third-party libraries like this in setup.py

Yes, it is acceptable but generally these should be minimized, especially if these are modules which have no obvious use for the end-user. Noone likes to have packages they don't need or use.

what is the recommended approach to using them?

There are basically 3 options:

• Bootstrap them (for example use pip to programmatically install packages). For example setuptools provides an ez_setup.py file that can be used to bootstrap setuptools. Maybe that can be customized to download and install appdirs.

• Include them (especially if it's a small package) in your project. For example appdirs is basically just a single file module. Pretty easy to copy and maintain in your project. Be very careful with licensing issues when you do that!

• Fail gracefully when it's not possible to import them and let the user install them. For example:

  try:
      import appdirs
  except ImportError:
      raise ImportError('this package requires "appdirs" to be installed. '
                        'Install it first: "pip install appdirs".')
  

Answer 2

You can mention install_requires with the dependencies list. Please check the python packaging guide here. Also you can provide a requirements.txt file so that it can be run at once using "pip install -r"

Answer 3

You could use pip to install the package programmatically if the import fails:

try:
    import appdirs
except ImportError:
    import pip
    pip.main(['install', 'appdirs'])
    import appdirs

In some circumstances you may need to use importlib or __import__ to import the package after pip.main or referesh the PATH variable. It could also be worthwhile to include a verification if the user really wants to install that package before installing it.

I used a lot of the examples from "Installing python module within code" and I haven't personally tried used this in setup.py files but it looks like it could be a solution for your question.