Python setuptools/pip packing data files into your package

Question

I have this git repo structure:

.gitignore
JSONs/subdirA/some.json
JSONs/subdirB/other.json
MyPackage/__init__.py
MyPackage/myModule.py

How d

Answer 1

Something like the following could help:

First we need to make sure that the json files are added to the source distribution.

MANIFEST.in:

recursive-include JSONs *.json

Then in the actual setup script, the list of packages has to be modified on the fly to take into account the target package structure.

setup.py:

#!/usr/bin/env python3

import setuptools

PACKAGES = (
    setuptools.find_packages(exclude=['JSONs*'])
    +
    [
        f'MyPackage.{package}'
        for package
        in setuptools.find_namespace_packages(include=['JSONs*'])
    ]
)

setuptools.setup(
    packages=PACKAGES,
    package_dir={
        'MyPackage.JSONs': 'JSONs',
    },
    include_package_data=True,
    #
    name='Something',
    version='1.2.3',
)

JSONs/subdirA/some.json :

{"Marco": "Polo"}

Such package data can be read like so:

MyPackage/myModule.py:

import pkgutil

print(pkgutil.get_data('MyPackage', 'JSONs/subdirA/some.json').decode())

And use it like in the following:

$ python -m pip install .
$ # Move to another directory to prevent that the current working directory
$ # ... overshadows the installed project
$ cd ..
$ python -m MyPackage.myModule
{"Marco": "Polo"}