Trouble installing scipy in virtualenv on a amazon ec2 linux micro instance

Question

I have successfully installed scipy in the default python compiler on an amazon ec2 micro instance (Ubuntu 13.04). However i am not able to install scipy<

Answer 1

Yes, 512MB is not enough for compiling that C++ file.

Your best option is to build Scipy as a binary package (bdist, or eggs, or, more modern wheels) e.g. via python setupegg.py bdist_egg on a different machine with compatible environment. For instance, use a similar Linux version to the EC2 instance in a virtual machine.

In general, it's good to remember that when pip installs packages, it compiles source files. If the package is not tiny, this is inefficient and it's better to use binary packages. The wheel package format is supposed to play well together with pip.

Answer 2

I was getting an out of virtual memory error during the scipy compilation on a t2.micro, I think numpy compiled just fine. Anyway, I guess that Dolan Antenucci's answer would fix my issue, but instead I went another route, a compromise, which does work.

I'm running an Ubuntu instance so keep that in mind.

sudo apt-get -y install python-scipy && echo -e "\nok, installed python-scipy, continuing...\n"

# add scipy to the venv
mkdir ~/venv_PROJECT/lib/python2.7/site-packages/scipy/
ln -s /usr/lib/python2.7/dist-packages/scipy/* ~/venv_PROJECT/lib/python2.7/site-packages/scipy/

# add numpy to the venv
mkdir ~/venv_PROJECT/lib/python2.7/site-packages/numpy/
ln -s /usr/lib/python2.7/dist-packages/numpy/* ~/venv_PROJECT/lib/python2.7/site-packages/numpy/

# add PIL to the venv
mkdir ~/venv_PROJECT/lib/python2.7/site-packages/PIL/
ln -s /usr/lib/python2.7/dist-packages/PIL/* ~/venv_PROJECT/lib/python2.7/site-packages/PIL/

The compromise is that you won't be able to use diferent versions among different virtual environments and you're tied to the version in the repo.

A word of warning:

>>> import scipy
>>> scipy.__version__
'0.13.3'

>>> import numpy
>>> numpy.__version__
'1.8.2'

>>> from PIL import Image
>>> Image.VERSION
'1.1.7'

Answer 3

One solution is to temporarily enable swap on your micro instance. As described at this SO post, enable 1gb swap via:

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo /sbin/swapon /var/swap.1

Once swap is on, install scipy via pip:

sudo apt-get install -y libatlas-base-dev gfortran python-dev build-essential g++
sudo pip install numpy
sudo pip install scipy

Once scipy successfully installs, you can disable it via:

sudo swapoff /var/swap.1
sudo rm /var/swap.1

Answer 4

This worked for me:

pip --no-cache-dir install scipy

See:

• Memory error while using pip install Matplotlib

  https://github.com/pypa/pip/blob/9a23d4ed119327d3b823ec223aaead90964bac58/pip/basecommand.py#L56-L63

• https://github.com/pypa/pip/blob/28cca11e284b37cc2c7977fd25be6f494adda9d3/src/pip/_internal/download.py#L359-L367

note:

• works for other service providers, hardware, VMs, and containers.
• if RAM allocation size of 1GB
• just calculate the diff between the cached directory memory usage and available ram

Answer 5

for me the error was slightly different.

Blas (http://www.netlib.org/blas/) libraries not found.

Directories to search for the libraries can be specified in the

numpy/distutils/site.cfg file (section [blas]) or by setting

the BLAS environment variable.

----------------------------------------
Cleaning up...
Command /home/kdixit/pyvirt/bin/python -c "import setuptools;__file__='/home/kdixit/pyvirt/build/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n

Thus I had to install

sudo apt-get install libblas-dev

and then it worked.