Check if my Python has all required packages

Question

I have a requirements.txt file with a list of packages that are required for my virtual environment. Is it possible to find out whether all the packages mention

Answer 1

You can run pip freeze to see what you have installed and compare it to your requirements.txt file.

If you want to install missing modules you can run pip install -r requirements.txt and that will install any missing modules and tell you at the end which ones were missing and installed.

Answer 2

If requirements.txt is like :

django
oursql
sys
notexistingmodule

Then the following script will tell you which modules are missing :

#!/usr/bin/python3
fname = 'requirements.txt'
with open(fname, 'r', encoding='utf-8') as fhd:
    for line in fhd:
        try:
            exec("import " + line)
        except:
            print("[ERROR] Missing module:", line)

This would print :

[ERROR] Missing module: notexistingmodule

Answer 3

UPDATE:

An up-to-date and improved way to do this is via distutils.text_file.TextFile. See Acumenus' answer below for details.

ORIGINAL:

The pythonic way of doing it is via the pkg_resources API. The requirements are written in a format understood by setuptools. E.g:

Werkzeug>=0.6.1
Flask
Django>=1.3

The example code:

import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict

# dependencies can be any iterable with strings, 
# e.g. file line-by-line iterator
dependencies = [
  'Werkzeug>=0.6.1',
  'Flask>=0.9',
]

# here, if a dependency is not met, a DistributionNotFound or VersionConflict
# exception is thrown. 
pkg_resources.require(dependencies)

Answer 4

Based on the answer by Zaur, assuming you indeed use a requirements file, you may want a unit test, perhaps in tests/test_requirements.py, that confirms the availability of packages.

Moreover, this approach uses a subtest to independently confirm each requirement. This is useful so that all failures are documented. Without subtests, only a single failure is documented.

"""Test availability of required packages."""

import unittest
from pathlib import Path

import pkg_resources

_REQUIREMENTS_PATH = Path(__file__).parent.with_name("requirements.txt")


class TestRequirements(unittest.TestCase):
    """Test availability of required packages."""

    def test_requirements(self):
        """Test that each required package is available."""
        # Ref: https://stackoverflow.com/a/45474387/
        requirements = pkg_resources.parse_requirements(_REQUIREMENTS_PATH.open())
        for requirement in requirements:
            requirement = str(requirement)
            with self.subTest(requirement=requirement):
                pkg_resources.require(requirement)

Answer 5

You can use the -r option from pip freeze that verifies that. It generates a WARNING log for packages that are not installed. One appropriated verbose mode should be selected in order the WARNING message to be shown. For example:

$ pip -vvv freeze -r requirements.txt | grep "not installed"

WARNING: Requirement file [requirements.txt] contains six==1.15.0, but package 'six' is not installed

Answer 6

In addition to check whether modules listed in requirements.txt are installed, you may want to check whether all used modules by your project are indeed listed in the requirements.txt file.

If you are using flake8 for style-checking, you can add flake8-requirements plugin for flake8. It will automatically check whether imported modules are available in setup.py, requirements.txt or pyproject.toml file. Additionally, for custom modules you can add custom configuration with known-modules (in order to prevent flake8 warnings). For more configuration options, see flake8-requirements project's description.