Trick Sam into building your Lambda Layers

Right now, the SAM CLI doesn’t support building Lambda Layers; those magical additions to Lambda that allow you to defined shared dependencies and modules. If you’re unfamiliar, you can read more about them here:

New for AWS Lambda – Use Any Programming Language and Share Common Components

If you read my last article on using sam build, you might think to yourself, “Hey, I can add Layers into my template to share common code across all my Lambdas!”, but hold on! At the moment sam build does not support building Layers the way it builds Lambda packages.

But, there is a hacky way around that. Here’s our repository:

carbon (7)

Now here’s the contents of template.yaml:

carbon (6).png

We’ve defined an AWS::Serverless::Function resource, but with no events, or any other attributes for that matter. We have also defined a AWS::Serverless::LayerVersion resource for our Lambda Layer, but the ContentUri path points to the build directory for the Lambda function.

See where this is going?

sam build will install all the dependencies for our Layer and copy its code into the build directory, and then when we call sam package the Layer will use that output! Spiffy. This does result in an orphan Lambda function that will never be used, but it won’t hurt anything just sitting out there.

Now, we aren’t done quite yet. According to AWS’s documentation, you need to place Python resources within a python directory inside your Layer. The zip file that sam build creates will be extracted into /opt, but the runtimes will only look, by default in a matching directory within /opt (so in the case of Python, that would be /opt/python).

See AWS Lambda Layers documentation for more details.

We can’t tell sam build to do that, but we can still get around this inside our Lambda functions that use the new Layer by adding /opt into sys.path (import searches all of the locations listed here when you call it). Here’s an example Python Lambda function that does this:

carbon (5)

Performing a test execution gives us the following output:

START RequestId: fd5a0bf2-f9af-11e8-bff4-ab8ada75cf17 Version: $LATEST
['/var/task', '/opt/python/lib/python3.6/site-packages', '/opt/python', '/var/runtime', '/var/runtime/awslambda', '/var/lang/lib/python36.zip', '/var/lang/lib/python3.6', '/var/lang/lib/python3.6/lib-dynload', '/var/lang/lib/python3.6/site-packages', '/opt/python/lib/python3.6/site-packages', '/opt/python', '/opt']
<module 'pymysql' from '/opt/pymysql/__init__.py'>
<module 'sqlalchemy' from '/opt/sqlalchemy/__init__.py'>
<module 'stored_procedures' from '/opt/stored_procedures.py'>
END RequestId: fd5a0bf2-f9af-11e8-bff4-ab8ada75cf17
REPORT RequestId: fd5a0bf2-f9af-11e8-bff4-ab8ada75cf17	Duration: 0.82 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 34 MB

Voila! We can see the inclusion of /opt into our path (and the expected path of /opt/python before it) and that our dependencies and custom module were all successfully imported.

It breaks PEP8 a little, but it gets the job done and we have now successfully automated the building and deployment of our Lambda Layer using AWS’s provided tooling.

 

 

Advertisements

3 thoughts on “Trick Sam into building your Lambda Layers

  1. This actually is supported now, works for me with samcli 0.9.0 and awscli 1.16

    AWSTemplateFormatVersion: 2010-09-09
    Transform: AWS::Serverless-2016-10-31
    Resources:
    Layer:
    Type: AWS::Serverless::LayerVersion
    Properties:
    CompatibleRuntimes:
    – nodejs8.10
    ContentUri: ./src/
    LayerName: foo-layer
    RetentionPolicy: Retain
    Outputs:
    LayerArn:
    Value: !Ref Layer

    In ./src, you put a runtime folder (like nodejs, python , whatever), then follow the AWS instructions from there as far as layer directory structure. sam build will copy template.yaml into .aws-sam/build, but will transform it such that ContentUri will still point to the src folder:

    ContentUri: ..\..\src

    Next, sam package (or more directly, aws cloudformation package) will create a zip archive of the contents of the src folder and upload them to an s3 bucket. This is why you need the new awscli, because it has the patch for the package step.

    Like

    • @DanLudwig this does not install third party packages. This blog post details using the existing `sam build` functionality to build the Lambda Layers using third party packages. Zipping the source URI into a usable layer was the supported functionality when I wrote this post.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s