Unable to install cvxpy into virtualenv for AWS lambda

问题

I am trying to run the cvxpy package in an AWS lambda function. This package isn't in the SDK, so I've read that I'll have to compile the dependencies into a zip, and then upload the zip into the lambda function.

I've done some research and tried out the links below, but when I try to pip install cvxpy I get error messages - I'm on a Windows box, but I know that AWS Lambda runs on Linux.

Appreciate the help!

http://i-systems.github.io/HSE545/machine%20learning%20all/cvxpy_install/CVXPY%2BInstallation%2BGuide%2Bfor%2BWindows.html

https://programwithus.com/learn-to-code/Pip-and-virtualenv-on-Windows/

https://medium.com/@manivannan_data/import-custom-python-packages-on-aws-lambda-function-5fbac36b40f8

https://www.cvxpy.org/install/index.html

回答1:

For installing cvxpy on windows it requires c++ build tools (please refer: https://buildmedia.readthedocs.org/media/pdf/cvxpy/latest/cvxpy.pdf)

On Windows:

• I created a lambda layer python directory structure python/lib/python3.7/site-packages (refer: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) and installed my pip packages in that site-packages directory.

pip install cvxpy --target python/lib/python3.7/site-packages

• Then, I zipped the python/lib/python3.7/site-packages as cvxpy_layer.zip and uploaded it to an S3 bucket (layer zipped file max limit is only 50 MB https://docs.aws.amazon.com/lambda/latest/dg/limits.html), to attach it to my lambda layers.
• Now, the layer is ready but the lambda is failing to import the packages as they were installed on a windows machine. (refer: AWS Lambda - unable to import module 'lambda_function')

On Linux:

• I created the same directory structure as earlier python/lib/python3.7/site-packages and installed the cvxpy and zipped it as shown below.
• Later I uploaded the zip file to an S3 bucket and created a new lambda layer.
• Attaching that lambda layer to my lambda function, I colud able to resolve the import issues failing earlier and run the basic cvxpy program on lambda.

mkdir -p alley/python/lib/python3.7/site-packages
pip install cvxpy --target alley/python/lib/python3.7/site-packages
cd alley
zip -rqvT cvxpy_layer.zip .

Lambda layer Image:

Lambda function execution:

回答2:

You can wrap all your dependencies along with lambda source into a single zipfile and deploy it. Doing this, you will end up having additional repetitive code in multiple lambda functions. Suppose, if more than one of your lambda functions needs the same package cvxpy, you will have to package it twice for both the functions individually.

Instead a better option would be to try Labmda Layers, where you put all your dependencies into a package and deploy a layer into your Lambda. Then attach that layer to your function to fetch its dependencies from there. The layers can even be versioned. :)

Please refer the below links:

• https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

• https://dev.to/vealkind/getting-started-with-aws-lambda-layers-4ipk

来源：https://stackoverflow.com/questions/58647020/unable-to-install-cvxpy-into-virtualenv-for-aws-lambda