Unable to import module 'lambda_function': No module named 'pandas'

Question

START RequestId: 3d5691d9-ad79-4eed-a26c-5bc3f1a23a99 Version: $LATEST Unable to import module \'lambda_function\': No module named \'pandas\'
END Request

Answer 1

I was able to import the pandas library successfully using a Lambda layer and an Amazon linux Cloud 9 instance. There are the commands I executed in the Cloud 9 instance and the Lambda function's output. I had to change the code slightly since it was failing with an import error and string value error.

Alternatively, these commands can also be executed in an EC2 instance. If it's not possible to use the SAM CLI(which uses docker) or just plain docker on windows we'll need to use an Amazon Linux instance to build everything since that's what AWS Lambda uses currently. I don't believe using an ubuntu instance will work here.

Commands:

python --version
Python 3.6.8

# https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
# python 3.6 uses Amazon Linux currently 

mkdir project
cd project
virtualenv v-env
source ./v-env/bin/activate
pip install pandas
deactivate

# creating layer
# https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path
mkdir python
cd python
cp -r ../v-env/lib64/python3.6/dist-packages/* .
cd ..
zip -r panda_layer.zip python
aws lambda publish-layer-version --layer-name pandas --zip-file fileb://panda_layer.zip --compatible-runtimes python3.6 

The publish-later-version command will create a new AWS Lambda layer in the region given in the command or the config file for the CLI.

A Lambda layer will apply the library to the Lambda function's code without needing to apply it directly to the deployment package. This also allows the use of the online code editor in Lambda since the deployment package is under the limit of 3MB

I applied the Lambda layer by clicking on the Layer button in the web console and choosing the layer version that I most recently published. I have a second version there because the first time I attempted this is put the contents of the lib directory which isn't for a 64 bit OS and my code failed in AWS Lambda.

Alternatively, you can also apply the layer using the CLI command update-function-configuration

Lambda function code I used:

import pandas as pd
import numpy as np

def lambda_handler(event, context):
    dates = pd.date_range(start='1/1/2018', end='1/08/2018')
    df = pd.DataFrame(np.random.randn(8, 4), index=dates, columns=list('ABCD'))
    print(str(df))

---

Lambda output:

START RequestId: 27f09b6c-a4cd-49dd-bb3e-ae6fc7cd5850 Version: $LATEST
                   A         B         C         D
2018-01-01 -1.040318  0.450841 -0.381687 -0.105480
2018-01-02 -1.381793 -0.481572  0.828419 -0.885205
2018-01-03  1.437799 -0.649816 -0.577112  0.400670
2018-01-04 -0.730997 -0.778775 -1.514203  1.165661
2018-01-05  1.963595 -1.137054  0.920218  0.960210
2018-01-06 -0.429179 -0.745549  1.482562  0.298623
2018-01-07 -1.082388 -0.529476 -1.051663  1.616683
2018-01-08  0.042779 -2.338471 -0.142992  0.680399
END RequestId: 27f09b6c-a4cd-49dd-bb3e-ae6fc7cd5850
REPORT RequestId: 27f09b6c-a4cd-49dd-bb3e-ae6fc7cd5850  Duration: 536.76 ms Billed Duration: 600 ms Memory Size: 128 MB Max Memory Used: 122 MB Init Duration: 1721.51 ms   
XRAY TraceId: 1-5d741e40-1311daa29fc16c74735988fc   SegmentId: 61a595dd3492c331 Sampled: false