Dockerfile: How to source Anaconda

问题

I am trying to set up an Anaconda environment with AWS Batch

Here is a snippet from my Dockerfile

#: Download Anaconda
COPY Anaconda3-2019.10-Linux-x86_64.sh /setup/
RUN bash Anaconda3-2019.10-Linux-x86_64.sh -b -p /home/ec2-user/anaconda3 
RUN echo 'export PATH=$PATH:/home/ec2-user/anaconda3/bin' >>~/.bashrc \
    && /bin/bash -c "source ~/.bashrc"
ENV PATH $PATH:/home/ec2-user/anaconda3/bin

# Create the environment:
COPY environment.yml .
RUN conda env create -f environment.yml

# Make RUN commands use the new environment:
SHELL ["conda", "run", "-n", "solver_env", "/bin/bash", "-c"]

When I access my container, and do some simple tests I get an error that

conda: command not found

Even though I have sourced the path to my bashrc file.

Also when trying to access my bashrc file I get

cat ~.bashrc: No such file or directory

and

cat: //.bashrc: No such file or directory

Any ideas on where I may be going wrong/where to check?

EDIT:

Last line in my Dockerfile to activate the venv and kick off the script is:

ENTRYPOINT ["conda", "run", "-n", "solver_env", "/bin/bash", "/usr/local/bin/fetch_and_run.sh"]

Once I have the image built and deployed to AWS ECR, I kick off a Batch job which essentially runs this shell script:

#!/bin/bash
date
echo "Args: $@"
env
echo "script_path: $1"
echo "script_name: $2"
echo "path_prefix: $3"
echo "jobID: $AWS_BATCH_JOB_ID"
echo "jobQueue: $AWS_BATCH_JQ_NAME"
echo "computeEnvironment: $AWS_BATCH_CE_NAME"
   
mkdir /tmp/scripts/
aws s3 cp $1 /tmp/scripts/$2
python3 /tmp/scripts/${@:2}

回答1:

You seem to have forgotten a slash in your command:

cat ~.bashrc: No such file or directory

It should be:

cat ~/.bashrc

Also everything you do with .bashrc in your Dockerfile is pointless. Not only, as @Itamar states in his answer is .bashrc only executed in interactive non-login shells, but every RUN command uses its own Bash process and you even use a separate process to source it manually, which essentially means you start Bash, source .bashrc and then discard that instance of Bash, discarding all changes with it.

I have answered a similar question on SuperUser and the solution applies here as well. It's actually simpler, since you want to use the Docker image with Docker and not with Singularity and you want to run is as root.

COPY Anaconda3-2019.10-Linux-x86_64.sh /setup/Anaconda3-2019.10-Linux-x86_64.sh
COPY environment.yml /setup/environment.yml
RUN bash /setup/Anaconda3-2019.10-Linux-x86_64.sh -b -p /home/ec2-user/anaconda3 && \
    rm /setup/Anaconda3-2019.10-Linux-x86_64.sh && \
    ln -s /home/ec2-user/anaconda3/bin/conda /usr/bin/conda && \
    conda env create -f /setup/environment.yml && \
    rm /setup/environment.yml

CMD ["conda", "run", "-n", "solver_env", "/bin/bash"]

If you want to run some more conda commands after creating the environment you should append them to the existing RUN instruction, like so:

COPY Anaconda3-2019.10-Linux-x86_64.sh /setup/Anaconda3-2019.10-Linux-x86_64.sh
COPY environment.yml /setup/environment.yml
RUN bash /setup/Anaconda3-2019.10-Linux-x86_64.sh -b -p /home/ec2-user/anaconda3 && \
    rm /setup/Anaconda3-2019.10-Linux-x86_64.sh && \
    ln -s /home/ec2-user/anaconda3/bin/conda /usr/bin/conda && \
    conda env create -f /setup/environment.yml && \
    rm /setup/environment.yml && \
    conda run -n solver_env python -V

CMD ["conda", "run", "-n", "solver_env", "/bin/bash"]

回答2:

Instead of modifying SHELL, you should instead prepend the environment's bin directory to PATH.

COPY Anaconda3-2019.10-Linux-x86_64.sh /setup/
RUN bash Anaconda3-2019.10-Linux-x86_64.sh -b -p /home/ec2-user/anaconda3 
RUN echo 'export PATH=$PATH:/home/ec2-user/anaconda3/bin' >>~/.bashrc \
    && /bin/bash -c "source ~/.bashrc"
ENV PATH=$PATH:/home/ec2-user/anaconda3/bin

# Create the environment:
COPY environment.yml .
RUN conda env create -f environment.yml
# Activate the environment.
ENV PATH=/home/ec2-user/anaconda3/envs/solver_env/bin:$PATH

回答3:

.bashrc is only used in interactive shells. Try adding -i to the ENTRYPOINT, before the -c.

来源：https://stackoverflow.com/questions/63996860/dockerfile-how-to-source-anaconda

标签

Docker

Anaconda

dockerfile