GeoDjango on Zappa / Lambda

Introduction

Zappa and AWS Lambda are great prototyping tools (and production tools too) but one tool that’s important to me has been missing from the drawer when I went to build something – GeoDjango. But all I could ever find were a few rough notes that involved compiling the two main libraries, GEOS and GDAL, inside specific Linux distros, bundling them in the app’s folder, and configuring them to be used. Straightforward, but a higher bar than I wanted to jump for free time projects. I would revisit every so often, and finally found this specific comment (https://github.com/Miserlou/lambda-packages/issues/12#issuecomment-478061924) which while not a tutorial was helpful and pretty comprehensive. Maybe there was a hack friendly solution.

The authors of the RasterIO Python package are already bundling GEOS and GDAL with their library. We can take advantage of that in a clever way I’ll document here, and have deployable geographic apps on Lambda in minutes.

First, add rasterio==1.0.25 to your requirements.txt and install from it.

Next, add the following to your environment_variables section of your zappa_settings.json:

"GDAL_LIBRARY_PATH": "/var/task/rasterio/.libs/libgdal-8b68ca59.so.20.5.2",
"GEOS_LIBRARY_PATH": "/var/task/rasterio/.libs/libgeos_c-595de9d4.so.1.10.2",

Third, configure your Django settings file to grab the newly configured settings:

GDAL_LIBRARY_PATH = os.environ.get('GDAL_LIBRARY_PATH')
GEOS_LIBRARY_PATH = os.environ.get('GEOS_LIBRARY_PATH’)

At this point, you should be able to "zappa update <env>" and have the libraries available, and be able to work with GeoDjango inside your application. (You will need a spatial database backend to take advantage of most of GeoDjango’s power, that discussion is out of scope for here)

Like many shortcuts, this isn’t the most efficient of approaches – Using RasterIO your project will be about 55MB before you even include your own app code. (Amazon allows up to 75MB for a Lambda function.) If you extract only the files needed from RasterIO’s .libs folder you’ll end up closer to 32MB. (This takes a good bit of trial and error, as the geographic libraries have a number of dependencies and to discover each meant doing a Lambda function release, watching logs to see what errors resulted, adding the next dependency, and re-deploying.)

Note: using Zappa’s slim_handler functionality to get around the code size restriction will change the paths GDAL_LIBRARY_PATH / GEOS_LIBRARY_PATH need to something more like /tmp//rasterio/.libs/libgeos_c-595de9d4.so.1.10.2 If you use a different version of RasterIO, or can’t get the paths above to work the "zappa invoke" command from the GitHub comment should find the file for you.

If you need a small app to test things out, you can use https://github.com/adamfast/geodjango-uscampgrounds which has importers / views for a (not recently updated) CSV dataset of campgrounds at http://uscampgrounds.info/takeit.html.