shebang: use interpreter relative to the script path

Question

I try to build scripts that work everywhere and always. For this I use a custom-built python, which is always in the parent directory relative to the script.

This way I could load my package on an USB-stick and it would work everywhere, regardless of where the stick is mounted and whether python is installed or not.

However, when I use

#!../python

then it works only when the script gets invoked from its directory, which is of course not acceptable.

Is there a way to do this or is this impossible in the current shebang-mechanism?

Answer 1

There is a healthy set of multi-line shebang scripts on this page for a lot of languages, example:

#!/bin/sh
"exec" "`dirname $0`/python" "$0" "$@"
print copyright

And if you want one-line shebang, this answer (and question) explains the issue in the details and suggests the following approaches using additional scripts inside the shebang:

Using AWK

#!/usr/bin/awk BEGIN{a=ARGV[1];sub(/[a-z_.]+$/,"python",a);system(a"\t"ARGV[1])}

Using Perl

#!/usr/bin/perl -e$_=$ARGV[0];exec(s/\w+$/python/r,$_)

Answer 2

Expanding @Anton's answer so as not to fail on spaces and other special characters in the path and to explain a bit more about the magic.

#!/bin/sh
"true" '''\'
exec "$(dirname "$(readlink -f "$0")")"/venv/bin/python "$0" "$@"
'''

__doc__ = """You will need to deliberately set your docstrings though"""

print("This script is interpretable by python and sh!")

This clever script is comprehendable by both sh and python. Each of which react to it differently. The first 2 lines after the shebang are interpreted by sh and cause it to hand over exec to a relative python binary (supplied with the same command line arguments). These same lines are safely discarded by python since they amount to a string ("true") followed by a multi-line string (''').

A little more about this subject can be read here.

Answer 3

For others who find this question while looking for a portable solution to the python hashbang line, and find the AWK command above does not work with passing multiple arguments, use the following instead.

#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i<ARGC;i++){b=b" \""ARGV[i]"\"";}sub(/[a-z_.\-]+$/,"python",a);system(a"\t"b)}

To change all the scripts hashbang lines in the current directory you can execute the following.

sed -i '1 s|^#!.*|#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i<ARGC;i++){b=b" \""ARGV[i]"\"";}sub(/[a-z_.\-]+$/,"python3.5",a);system(a"\\t"b)}|' *

Answer 4

After looking at these answers, I decided to use a Python-specific solution that does not actually change the shebang.

This solution uses the system python interpreter to find the desired interpreter and execute that. This is useful to me because it allows me to munge environment variables as well as ensuring the correct interpreter.

Because it uses exec, it does not double the memory use-- the new interpreter replaces the last one. Also, exit status and signals will be handled correctly.

It is a python module that is loaded by the script that needs to run under this environment. Importing this module has the side-effect of launching a new interpreter if needed. Generally, modules should not have side-effects, but the alternative is to run the module's function before non-system imports are performed, and that would be a PEP8 violation. So you have to pick your poison.

"""Ensure that the desired python environment is running."""


import os
import sys


def ensure_interpreter():
    """Ensure we are running under the correct interpreter and environ."""
    abs_dir = os.path.dirname(os.path.abspath(__file__))
    project_root = os.path.normpath(os.path.join(abs_dir, '../../../'))
    desired_interpreter = os.path.join(project_root, 'bin/python')

    if os.path.abspath(sys.executable) == desired_interpreter:
        return

    env = dict(os.environ)

    def prefix_paths(key, prefix_paths):
        """Add prefix paths, relative to the project root."""
        new_paths = [os.path.join(project_root, p) for p in prefix_paths]
        new_paths.extend(env.get(key, '').split(':'))
        env[key] = ':'.join(new_paths)

    prefix_paths('PYTHONPATH', ['dir1', 'dir2'])
    prefix_paths('LD_LIBRARY_PATH', ['lib'])
    prefix_paths('PYTHON_EGG_CACHE', ['var/.python-eggs'])
    env['PROJECT_ROOT'] = project_root
    os.execvpe(desired_interpreter, [desired_interpreter] + sys.argv, env)


ensure_interpreter()

If you don't need to munge any environment variables, you can strip out everything between env = dict(os.environ) and os.execvpe(desired_interpreter, [desired_interpreter] + sys.argv, env).