AWS Lambda Layers - What Are They Good For?

There’s no doubt that AWS Lambda and Function as a Service (FAAS) have been game changers in the microservice architecture landscape. AWS Lambda simplifies deployments using SAM or CloudFormation and allows for easy CI/CD pipeline (I'll discuss the in more detail in a future post). Serverless deployments are a fraction of the cost of traditional servers or EC2 instances and have much less infrastructure overhead, leading to a quicker project ramp up time. Utilizing serverless effectively allows you to focus on what matters most: robust, quality code and application logic. Serverless also works very well within any microservice architecture.

A powerful extension of AWS Lambda and less-talked-about feature is Layers. In this post we’ll explore the following.

1. What are Lambda Layers?
2. What are they good for?
3. An example use case - Layers to the Rescue!

What are Lambda Layers?

Lambda Layers are a way to share code and external dependencies between Lambdas within and across different AWS accounts. Layers were introduced at re:invent 2018 and have been a less talked about feature, but they deserve a closer look if you’re going to be using Lambda.

Layers are code, data or dependencies packaged separately for use within your lambda functions. Dependent layers are unpacked in the deployed lambda’s /opt directory for use. Layers are also versioned by default which allows you to make breaking changes in new layer versions and only reference the new version when ready.

What are they good for?

The biggest strength of Layers is the ability to keep your project structure and repository leaner by extracting large, less frequently updated dependencies out into their own layer. Some project dependencies may require a separate, more complicated build process and can add complexity or slowness to your build. Those larger dependencies are an obvious candidate for their own layer.

A recent project requirement that fit well with Layers

On a recent project we had a requirement to convert image files from an Adobe Illustrator format to a web friendly format on the fly. This solution had to fit nicely into our existing serverless architecture. A common way to convert these files programmatically is using the free and open source application ImageMagick.

One approach would have been to create a small API project, package the application with Docker for easier deployments and the ImageMagick install, and have it deployed and running, ready to serve requests and do file conversions. Easy enough, right? This approach is wasteful, having a costly server always running and charging by the second to wait for requests… that’s where Lambdas excel, right? Cutting costs by not running all the time?

To make ImageMagick accessible to our lambda function was not as simple as running an install command. The prebuilt binaries would need to be packaged and zipped up with our lambda code. This would then require downloading the ImageMagick source on an Amazon Linux AMI that matches AWS Lambda's runtime, running some makefile build steps and adding the build output to a directory within our project. This would overcomplicate and slow our build process and even doing this once and adding the output would still bloat the project repository.

Layers and the Serverless Application Repository to the rescue!

What is the Serverless Application Repository?

The Serverless Application Repository is a place to share serverless applications and functionality, but the home page describes it best:

“The AWS Serverless Application Repository is a managed repository for serverless applications. It enables teams, organizations, and individual developers to store and share reusable applications, and easily assemble and deploy serverless architectures in powerful new ways. Using the Serverless Application Repository, you don't need to clone, build, package, or publish source code to AWS before deploying it. Instead, you can use pre-built applications from the Serverless Application Repository in your serverless architectures, helping you and your teams reduce duplicated work, ensure organizational best practices, and get to market faster.”

A quick search of the Serverless Application Repository revealed ImageMagick and GhostScript Layers (extra dependency required for .ai Adobe Illustrator file conversion) ready to use. All the steps to build and reference ImageMagick from the Lambda became a few simple CloudFormation entries.

Entries to create the layers within our account:

We can then reference the layers with our Lambda:

Using ImageMagick becomes a few lines of code using a helpful NPM library:

NOTE
Without the NPM imagemagick module, our layer files would still be accessible from the /opt/bin/convert directory and could be invoked with Node’s child_process command.

Wrap up

There are a few limitations to keep in mind, A Lambda function can only use up to 5 layers at a time and the total unzipped size of the function and all layers cannot exceed the unzipped deployment package size limit of 250 MB.

This post only scratches the surface of the potential of Lambda Layers, but hopefully showcases their potential for code reuse. Thinking about the different ways to structure your serverless applications, leveraging existing serverless applications from an open public repository can speed up development and improve overall quality.