Pyinstaller on a setuptools package

Question

I\'m attempting to run PyInstaller on a CLI app I am building in Python using the Click library. I\'m having trouble building the project using PyInstaller. PyInstaller has

Answer 1

First: I used a combination of Stephen's answer, and some digging of my own to find the answer. In the end, Stephen's first part did the trick: manually adding / exporting the PYTHONPATH variable. You can actually specify this using pathex in the Entrypoint function like so:

a = Entrypoint('myapp-cli',
    'console_scripts',
    'myapp',
    pathex=['/some/path/to/myapp-cli/myapp', '/some/path/to/myapp-cli']
)

I didn't end up needing the myapp.main after all.

Second: I was still having issues with PyInstaller not producing a single binary. For me, this did the trick:

• Add the latest version of PyInstaller to your requirements.txt or to your install_requires in setup.py: https://github.com/pyinstaller/pyinstaller/archive/develop.zip.
• Also, you can make your .spec file with the --onefile option in pyi-makespec like so: pyi-makespec --onefile myapp.py. This will make a .spec file that ensures that all of your packages are compiled into the binary.

In the end, the following spec file did the trick, and I was able to make a fully working binary:

# -*- mode: python -*-

block_cipher = None

def Entrypoint(dist, group, name,
               scripts=None, pathex=None, hiddenimports=None,
               hookspath=None, excludes=None, runtime_hooks=None):
    import pkg_resources

    # get toplevel packages of distribution from metadata
    def get_toplevel(dist):
        distribution = pkg_resources.get_distribution(dist)
        if distribution.has_metadata('top_level.txt'):
            return list(distribution.get_metadata('top_level.txt').split())
        else:
            return []

    hiddenimports = hiddenimports or []
    packages = []
    for distribution in hiddenimports:
        packages += get_toplevel(distribution)

    scripts = scripts or []
    pathex = pathex or []
    # get the entry point
    ep = pkg_resources.get_entry_info(dist, group, name)
    # insert path of the egg at the verify front of the search path
    pathex = [ep.dist.location] + pathex
    # script name must not be a valid module name to avoid name clashes on import
    script_path = os.path.join(workpath, name + '-script.py')
    print ("creating script for entry point", dist, group, name)
    with open(script_path, 'w') as fh:
        print("import", ep.module_name, file=fh)
        print("%s.%s()" % (ep.module_name, '.'.join(ep.attrs)), file=fh)
        for package in packages:
            print ("import", package, file=fh)

    return Analysis([script_path] + scripts, pathex, hiddenimports, hookspath, excludes, runtime_hooks)

a = Entrypoint('myapp-cli',
    'console_scripts',
    'myapp',
    pathex=['/some/path/to/myapp-cli/myapp', '/some/path/to/myapp-cli']
)

pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='myapp',
          debug=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )

I think in the end using something like Cobra for Golang would work easier since Golang compiles one-file binaries out of the box. However, if you prefer Python, this should do the trick.

Answer 2

After much searching, this error is generally due to attempting to access your project's package's metadata (i.e. version being the primary one).

Package metadata is typically accessed using pkg_resources or the older distutil, either explicitly, or often hidden in other packages (usually attempting to access the package version). Starting with Python v3.8 it will also be available in the stdlib within importlib.metadata.

If this is the case, you likely need to include some or all of the files in a mypackage.egg-info folder, especially the file PKG_INFO, but it may require them all.

There are multiple ways to do this, here are several I like:

---

1. If you are using a script.spec file, you can update the datas= line to include this info, per Charles answer:

a = Analysis(['myscript.py'],
             pathex=['C:\\path\\to\\mypackage'],
             binaries=[],
             datas=[('mypackage.egg-info/*','mypackage.egg-info')],

---

2. Create a custom hook file, put it in a directory, and add the directory as a custom hooks directory at the command line.

Create a hook-mypackage.py hook file, with the following very simple, and pretty elegant, lines:

from PyInstaller.utils.hooks import copy_metadata

datas = copy_metadata('md2mat')

I put this into a new hooks folder in my root package/repo folder, then added the following to my pyinstaller command:

pyinstaller -F -y --additional-hooks-dir=hooks myscript.py

It works pretty well, and assuming the copy_metadata function is well maintained as we change over from older metadata packages to the new importlib.metadata, it should work well through future Python updates.

---

3. Add additional data files directly at the command-line

This might be my favorite, if I could get it working...

pyinstaller --add-data <SRC;DEST> myscript.py

This option --add-data shows up in the help output (pyinstaller --help), and indicates the format of the argument should be SRC;DEST for Windows, so I think it must match the datas= format from the other methods, but I couldn't get it to work.

The closest I think I got to the right format were the following:

pyinstaller -F -y --add-data "mypackage.egg-info/*;mypackage.egg-info"
pyinstaller -F -y --add-data="mypackage.egg-info/*;mypackage.egg-info"

These would compile, but the resulting exe would just run with no output.

The --add-data option is missing from the PyInstaller Documentation, but shows up when running pyinstaller --help-commands.

Answer 3

This error:

pkg_resources.DistributionNotFound: The 'myapp' distribution was not found and is required by the application

indicates that this package is not on PYTHONPATH. I fixed it on Windows with:

set PYTHONPATH=.

adjust to your OS of choice.

---

In addition to the path problem, there is:

In setup.py:

setup(
    entry_points = '''
        [console_scripts]
        myapp=myapp.main:entry_point
    ''',

In main.spec:

a = Entrypoint('myapp', 'console_scripts', 'myapp')

According to setup.py, it looks like your entry point is myapp.main not myapp. So you may need:

a = Entrypoint('myapp', 'console_scripts', 'myapp.main')

Answer 4

Something I've noticed is that the typical way of adding a data file doesn't work once you've monkey patched Entrypoint in the way Scott Crooks recommends in the ticked answer. For me, I had to append to the a.datas array. In python3, this looks like:

...
a = Entrypoint(...)
from pathlib import Path
Path('/tmp/modulename/datafile.txt').write_text(Path('datafile.txt').read_text()))
a.datas.append('datafile.txt', '/tmp/modulename/datafile.txt', 'DATA')

pyz = PYZ(...)
...

Answer 5

The accepted answer didn't work for me. I had to add the egg-info directory via the .spec file.

My call to the Entrypoint function looks like this:

a = Entrypoint(
        'PrintIt',
        'console_scripts',
        'printit',
        datas=[('plugins/*.egg', 'plugins/'),
               ('../PrintIt.egg-info/*', 'PrintIt.egg-info/')])