Python - Activate conda env through shell script

Question

I am hoping to run a simple shell script to ease the management around some conda environments. Activating conda environments via conda activate in a lin

Answer 1

Using conda activate or source activate in shell scripts does not always work and can throw errors like this. An easy work around it to place source ~/miniconda3/etc/profile.d/conda.sh above any conda activate command in the script:

source ~/miniconda3/etc/profile.d/conda.sh # Or path to where your conda is
conda activate some-conda-environment

This is the solution that has worked for me and will also work if sharing scripts. This also get around having to use conda init as on some clusters I have worked with the system is initialised but conda activate still wont work in a shell script.

Answer 2

The error message is rather helpful - it's telling you that conda is not properly set up from within the subshell that your script is running in. To be able to use conda within a script, you will need to (as the error message says) run conda init bash (or whatever your shell is) first. The behaviour of conda and how it's set up depends on your conda version, but the reason for the version 4.4+ behaviour is that conda is dependent on certain environment variables that are normally set up by the conda shell itself. Most importantly, this changelog entry explains why your conda activate and deactivate commands no longer behave as you expect in versions 4.4 and above.

For more discussion of this, see the official conda issue on GitHub.

---

Edit: Some more research tells me that the conda init function mentioned in the error message is actually a new v4.6.0 feature that allows a quick environment setup so that you can use conda activate instead of the old source activate. However, the reason why this works is that it adds/changes several environment variables of your current shell and also makes changes to your RC file (e.g.: .bashrc), and RC file changes are never picked up in the current shell - only in newly created shells. (Unless of course you source .bashrc again). In fact, conda init --help says as much:

IMPORTANT: After running conda init, most shells will need to be closed and restarted for changes to take effect

However, you've clearly already run conda init, because you are able to use conda activate interactively. In fact, if you open up your .bashrc, you should be able to see a few lines added by conda teaching your shell where to look for conda commands. The problem with your script, though, lies in the fact that the .bashrc is not sourced by the subshell that runs shell scripts (see this answer for more info). This means that even though your non-login interactive shell sees the conda commands, your non-interactive script subshells won't - no matter how many times you call conda init.

This leads to a conjecture (I don't have conda on Linux myself, so I can't test it) that by running your script like so:

bash -i shell_script.sh

you should see conda activate work correctly. Why? -i is a bash flag that tells the shell you're starting to run in interactive mode, which means it will automatically source your .bashrc. This should be enough to enable you to use conda within your script as if you were using it normally.

Answer 3

I use 'source command' to run the shell script, it works:

source shell_script.sh

Answer 4

Quick solution for bash: prepend the following init script into your Bash scripts.

eval "$(command conda 'shell.bash' 'hook' 2> /dev/null)"

Done.

---

For other shells, check the init conf of your shell, copy the following content within the shell conf and prepend it into your scripts.

# >>> conda initialize >>>
...
# <<< conda initialize <<<

You can also use

conda init --all --dry-run --verbose

to get the init script you need in your scripts.

Explanation

This is related with the introduction of conda init in conda 4.6.

Quote from conda 4.6 release log

Conda 4.4 allowed “conda activate envname”. The problem was that setting up your shell to use this new feature was not always straightforward. Conda 4.6 adds extensive initialization support so that more shells than ever before can use the new “conda activate” command. For more information, read the output from “conda init –help”

After conda init is introduced in conda 4.6, conda only expose command conda into the PATH but not all the binaries from "base". And environment switch is unified by conda activate env-name and conda deactivate on all platforms.

But to make these new commands work, you have to do an additional initialization with conda init.

The problem is that your script file is run in a sub-shell, and conda is not initialized in this sub-shell.

References

• Conda 4.6 Release
• Unix shell initialization
• Shell startup scripts

Answer 5

I also had the exact same error when trying to activate conda env from C++ or Python file. I solved it by bypassing the conda activate statement and using the absolute path of the specific conda env.

For me, I set up an environment called "testenv" using conda.

I searched all python environments using

whereis python | grep 'miniconda'

It returned a list of python environments. Then I ran my_python_file.py using the following command.

~/miniconda3/envs/testenv/bin/python3.8 my_python_file.py

You can do the same thing on windows too but looking up for python and conda python environments is a bit different.

Answer 6

What is the problem with simply doing something like this in your shell:

source /opt/conda/etc/profile.d/conda.sh

(The conda init is still marked as Experimental, and thus not sure if it is a good idea to use it yet).