Excluding source files from built rpm distribution with setuptool

问题

I have a typical project structure that looks as follows:

EngineEmulator
    src
        ship
            engine
                emulator
                mapping
            tests
                emulator
                mapping
        utils
            common

    doc
       ....
    tools
       ....
    setup.py
    MANIFEST.in
    setup.cfg
    README.rst

My setup.py looks as follows:

from setuptools import setup, find_packages
setup(
   name='Engine',
   version=1.0.0,
   description='Engine Project',       
   package_dir={'': 'src'},
   packages=find_packages(
    'src',
    exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
   install_requires =['pycrypto', 
                      'kombu >=1.1.3'],
   author='Demo',
   author_email='demo@eliza.net'
   license='MIT',
   classifiers=[
    'Topic :: Demo Engine',
    'Development Status:: 3 - Iteration',
    'Programming Language :: Python -2.6'
]

)

My setup.cfg looks as follows:

[egg_info]
tag_build = .dev
tag_svn_revision = 1

[rotate]
#keep last 15 eggs, clean up order
match = .egg
keep = 15   

And My MANIFEST.in looks as follows:

include README.rst
recursive-include src/ship/Engine
prune src/utils
prune src/ship/tests
prune tools/

When I run python setup.py bdist_egg and python setup.py bdist_rpm I get the egg file and two rpm files generated (noarch.rpm and src.rpm).

In my destination machine when I run easy_install <generated egg file> my eg.info file gets copied over but the source files don't get copied over to /usr/lib/python2.6/site-packages. I was expecting I would have a directory called Engine.

Can anybody point out what I am doing wrong? Thanks in advance.

回答1:

Try to keep things as simple as possible.

Quick check with sdist

Try this:

$ python setup.py sdist

It shall create source distribution file for your package.

It is in zip format, so unpack it and check, if there are all expected files inside present.

If not, you have to find the reason, why expected files are missing in your distribution.

Checking things step by step (and simplifying)

Do you use .py extension?

May be stupid question, but in your file listing I do not see any py files inside of src tree.

In case you have there just files without .py extension, find_packages will not find anything.

Where do you have your __init__.py files located?

Let us know, where the files are:

$ cd src
$ find . -name "*.py"

If you miss __init__.py, find_packages will not find whole package.

Remove utils package

Why do you have it there?

Better have it installed out of your source code you develop or move it subdirectory in your project root.

This will render prune src/utils unnecessary in your MANIFEST.in.

Put into MANIFEST.in only what must be there

If you read doc for MANIFEST.in, it states, what files are included automatically (all, what mentioned in arguments of setup function, so in your case all python source files returned by find_packages).

For this reason, you shall remove recursive-include src/shop/Engine as it shall be already included by setup call.

Remove prune lines.

• src/utils shall not be in your source tree - it is just messing things up.
• tools is not to be included, so there is no need to prune it.
• src/ship/tests can be there, it will not harm, if you keep these files in the destribution.

Assert, what packages were found

Make sure, your setup get proper names for packages.

For this purpuse, you can call find_package sooner and assert it containts, what you expect.

(temporarily) remove setup.cfg

Just to keep things simpler.

Proposed project reorganization

You shall have file structure in similar manner as follows:

src/ship/__init__.py
src/ship/engine/__init__.py
src/ship/engine/emulator/__init__.py
src/ship/engine/emulator/module.py
src/ship/engine/emulator/module2.py
src/ship/engine/mapping/other.py
src/ship/engine/mapping/another.py
src/ship/tests/__init__.py
src/ship/tests/emulator/__init__.py
src/ship/tests/emulator/test_module.py
src/ship/tests/emulator/test_module2.py
src/ship/tests/mapping/__init__.py
src/ship/tests/mapping/test_other.py
src/ship/tests/mapping/test_another.py
doc
doc/index.rst
tools
tools/knife.py
setup.py
MANIFEST.in
README.rst

setup.py

from setuptools import setup, find_packages

packages=find_packages("src")
assert "ship.engine" in packages
assert "ship.engine.emulator" in packages
assert "ship.engine.mapping" in packages
#etc

install_requires =['pycrypto', 'kombu>=1.1.3'] #watch the spaces around `>=`, shall not be there

setup(
    name="Engine",
    package_dir={'': 'src'},
    packages=packages,
    install_requires=install_requires
)

MANIFEST.in

include README.rst

Conclusions

It might happen, that running

$ python setup.py sdist

would fail on asserts. This is sign, some of expected files are missing. Check that.

After you make your project living in simple way, you might add more details around (and do it step by step to be sure, you do not break something).

来源：https://stackoverflow.com/questions/24924296/excluding-source-files-from-built-rpm-distribution-with-setuptool