docker - AWS lambda container not picking system dependency

Question

I am Building a lambda to denoise audio files. Python soundfile uses libsndfile system dependency which I am installing via apt in my DockerFile. The container is running fine locally but when I run it after deploying to lambda it says [ "errorMessage": "sndfile library not found", "errorType": "OSError", ]. Here is my Dockerfile,

# Define global args
ARG FUNCTION_DIR="/home/app"
ARG RUNTIME_VERSION="3.7"

# Stage 1 - bundle base image + runtime
# Grab a fresh copy of the image and install GCC if not installed ( In case of debian its already installed )
FROM python:${RUNTIME_VERSION} AS python-3.7

# Stage 2 - build function and dependencies
FROM python-3.7 AS build-image
# Install aws-lambda-cpp build dependencies ( In case of debian they're already installed )
RUN apt-get update && apt-get install -y 
    g++ 
    make 
    cmake 
    unzip 
    libcurl4-openssl-dev
# Include global args in this stage of the build
ARG FUNCTION_DIR
ARG RUNTIME_VERSION
# Create function directory
RUN mkdir -p ${FUNCTION_DIR}
# Copy handler function
COPY app/requirements.txt ${FUNCTION_DIR}/app/requirements.txt
# Optional – Install the function's dependencies
RUN python${RUNTIME_VERSION} -m pip install -r ${FUNCTION_DIR}/app/requirements.txt --target ${FUNCTION_DIR}
# Install Lambda Runtime Interface Client for Python
# RUN python${RUNTIME_VERSION} -m pip install awslambdaric --target ${FUNCTION_DIR}

# Stage 3 - final runtime image
# Grab a fresh copy of the Python image
FROM python-3.7
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the built dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}
# Install librosa system dependencies
RUN apt-get update -y && apt-get install -y 
    libsndfile1 
    ffmpeg
# (Optional) Add Lambda Runtime Interface Emulator and use a script in the ENTRYPOINT for simpler local runs
# ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
# COPY entry.sh /
COPY app ${FUNCTION_DIR}/app
ENV NUMBA_CACHE_DIR=/tmp
RUN ln -s /usr/lib/x86_64-linux-gnu/libsndfile.so.1 /usr/local/bin/libsndfile.so.1
# enable below for local testing
# COPY events ${FUNCTION_DIR}/events
# COPY .env ${FUNCTION_DIR}
# RUN chmod 755 /usr/bin/aws-lambda-rie /entry.sh
ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
CMD [ "app.handler.lambda_handler" ]

Below is my lambda config

{
    "FunctionName": "DenoiseAudio",
    "FunctionArn": "arn:aws:lambda:us-east-1:xxxx:function:DenoiseAudio",
    "Role": "arn:aws:iam::xxxx:role/lambda-s3-role",
    "CodeSize": 0,
    "Description": "",
    "Timeout": 120,
    "MemorySize": 128,
    "LastModified": "2021-01-25T13:41:00.000+0000",
    "CodeSha256": "84ae6e6e475cad50ae5176d6176de09e95a74d0e1cfab3df7cf66a41f65e4e19",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "43c6e7c4-27a8-4c6d-8c32-c1e074d40a62",
    "State": "Active",
    "LastUpdateStatus": "Successful",
    "PackageType": "Image",
    "ImageConfigResponse": {}
}

question from:https://stackoverflow.com/questions/65886454/aws-lambda-container-not-picking-system-dependency

Answer 1

It looks like you are not building from one of the AWS managed base images which include the runtime interface client. It also looks like you are not adding it manually as well. My recommendation is to start with the Python3.7 or Python3.8 base image (both found here). Then add the other libraries as needed. I would also encourage you to look at the AWS SAM implementation of container image support for Lambda functions. It will make your life easier :). You can find a video demonstration of it here: https://youtu.be/3tMc5r8gLQ8?t=1390.

If you want still want to build an image from scratch, take a look at the documents that will guide you through the requirements found here.