How to activate an Anaconda environment in a Singularity recipe

问题

I am trying to create a singularity image and recipe that will create an anaconda environment and then activate said environment so I can build the python wheel of a project in that environment so it's 100% installed and functional after the singularity build is completed.

Bootstrap: docker
From: nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04

%environment

  # use bash as default shell
  SHELL=/bin/bash

  # add CUDA paths
  CPATH="/usr/local/cuda/include:$CPATH"
  PATH="/usr/local/cuda/bin:$PATH"
  LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
  CUDA_HOME="/usr/local/cuda"

  # add Anaconda path
  PATH="/usr/local/anaconda3/bin:$PATH"

  export PATH LD_LIBRARY_PATH CPATH CUDA_HOME
  export MKL_NUM_THREADS=1
  export OPENBLAS_NUM_THREADS=1

%setup
  # runs on host
  # the path to the image is $SINGULARITY_ROOTFS

%post
  # post-setup script

  # load environment variables
  . /environment

  # use bash as default shell
  echo "\n #Using bash as default shell \n" >> /environment
  echo 'SHELL=/bin/bash' >> /environment

  # make environment file executable
  chmod +x /environment

  # default mount paths
  mkdir /scratch /data 

  #Add CUDA paths
  echo "\n #Cuda paths \n" >> /environment
  echo 'export CPATH="/usr/local/cuda/include:$CPATH"' >> /environment
  echo 'export PATH="/usr/local/cuda/bin:$PATH"' >> /environment
  echo 'export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"' >> /environment
  echo 'export CUDA_HOME="/usr/local/cuda"' >> /environment

  # updating and getting required packages
  apt-get update
  apt-get install -y wget git vim build-essential cmake

  # creates a build directory
  mkdir build
  cd build

  # download and install Anaconda
  CONDA_INSTALL_PATH="/usr/local/anaconda3"
  wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
  chmod +x Anaconda3-5.0.1-Linux-x86_64.sh
  ./Anaconda3-5.0.1-Linux-x86_64.sh -b -p $CONDA_INSTALL_PATH

  # download and install CaImAn
  git clone https://github.com/flatironinstitute/CaImAn.git
  cd CaImAn
  conda env create -n caiman -f environment.yml
  source activate caiman
  pip install .
  caimanmanager.py install
  source deactivate


%runscript
  # executes with the singularity run command
  # delete this section to use existing docker ENTRYPOINT command

%test
  # test that script is a success

I've tried both conda activate and source activate and get the same error for both.

+ source activate caiman
/bin/sh: 41: source: not found
ABORT: Aborting with RETVAL=255
Cleaning up...

Is this just something I have to do afterwards by making the image writable? That would be the next default solution, but it would be nice if the recipe could just work.

*Edit 1 . activate caiman returns.

+ . activate caiman
+ [[ -n ]]
/bin/sh: 4: /usr/local/anaconda3/bin/activate: [[: not found
+ [[ -n ]]
/bin/sh: 7: /usr/local/anaconda3/bin/activate: [[: not found
+ echo Only bash and zsh are supported
Only bash and zsh are supported
+ return 1
ABORT: Aborting with RETVAL=255
Cleaning up...

*Edit 2 By using a newer version of Anaconda, the not found error goes away. All I did was change the Anaconda distribution I got with wget, and I also forced and update just to be doubly sure.

# download and install Anaconda
  CONDA_INSTALL_PATH="/usr/local/anaconda3"
  wget https://repo.continuum.io/archive/Anaconda3-5.3.1-Linux-x86_64.sh
  chmod +x Anaconda3-5.3.1-Linux-x86_64.sh
  ./Anaconda3-5.3.1-Linux-x86_64.sh -b -p $CONDA_INSTALL_PATH
  conda update -n base -c defaults conda
  pip install --upgrade pip

回答1:

If I am not wrong (which is totally possible) the same happens with virtualenv.

The problem is source is not a command, try:

. activate caiman

instead of

source activate caiman

Editing after updated question, check this https://github.com/conda/conda/issues/6639 you might want to investigate what your activate is doing (seems to be looking for non existing files)

来源：https://stackoverflow.com/questions/55168018/how-to-activate-an-anaconda-environment-in-a-singularity-recipe

标签

python

Anaconda

singularity-container