Skip to main content

Compiles Python packages on Amazon Linux to ensure their compatability when used with AWS Lambda.

Project description

Amazon Linux Python Package Builder (alppb)

alppb builds Python packages using the same version of Amazon Linux that the AWS Lambda service uses. Using alppb helps guarantee that any PyPi package your AWS Lambda app depends on will run properly.

Why is this a problem that needs to be solved? AWS Lambda requires you to package up your Python project along with all of its dependencies in order to run. If your AWS Lambda Python project has package(s) with C extension modules (or dependencies that do), you will need to build them on Amazon Linux for your app to work. alppb uses the AWS CodeBuild service (perpetual free tier includes 100 build minutes per month) to build the package(s) on Amazon Linux and download them to your local machine for you. Simply unzip the downloaded package(s) into your deployment bundle and upload to the AWS Lambda service.

How To Use alppb

pip install alppb
alppb -h

Build package requests in bucket foo alppb requests foo

TODO

Pre 1.0.0

  • <input type="checkbox" disabled="" /> Foundation - create a CodeBuild project with hardcoded build that puts an artifact in s3
  • <input type="checkbox" disabled="" /> Fix artifact so its a zip of the contents (excluding parent dir)
  • <input type="checkbox" disabled="" /> Download the module locally to dir alppb was run from
  • <input type="checkbox" disabled="" /> Move codebuild stuff to a module
  • <input type="checkbox" disabled="" /> Delete the artifact from s3 as part of cleanup
  • <input type="checkbox" disabled="" /> Add creation of IAM role for CodeBuild instead of using hardcoded, pre-built role
  • <input type="checkbox" disabled="" /> Add deletion of IAM role as part of cleanup
  • <input type="checkbox" disabled="" /> Move aws-cli stuff to boto3
  • <input type="checkbox" disabled="" /> Allow user specification of the desired module to be built using alppb
  • <input type="checkbox" disabled="" /> Cleanup existing docstrings
  • <input type="checkbox" disabled="" /> Remove base64 stuff in iam.py as it obscures whats happening
  • <input type="checkbox" disabled="" /> Axe the examples dir
  • <input type="checkbox" disabled="" /> Allow user specification of the bucket

1.0.0

  • <input type="checkbox" disabled="" /> Exception handling
    • <input type="checkbox" disabled="" /> Update and overwrite if resources already exist
      • <input type="checkbox" disabled="" /> pre-req checking
      • <input type="checkbox" disabled="" /> Valid PyPi package
      • <input type="checkbox" disabled="" /> Bucket and CodeBuild need to be in same region
      • <input type="checkbox" disabled="" /> Bucket exists (NoSuchBucket)
      • <input type="checkbox" disabled="" /> Bucket has valid name (botocore.exceptions.ParamValidationError)
  • <input type="checkbox" disabled="" /> Unit tests
  • <input type="checkbox" disabled="" /> Integration tests
    • <input type="checkbox" disabled="" /> Test each version of Python supported
      • <input type="checkbox" disabled="" /> Verify they're using the actual right python versions as part of each test
      • <input type="checkbox" disabled="" /> Inspect the zip and make sure it contains what's expected
  • <input type="checkbox" checked="" disabled="" /> Package and Submit to PyPi
  • <input type="checkbox" checked="" disabled="" /> Make CodeBuild Docker image details more clear and document
  • <input type="checkbox" disabled="" /> Add verbosity levels
  • <input type="checkbox" disabled="" /> Add Sphinx docs
    • <input type="checkbox" disabled="" /> readthedocs.org

Planned

  • <input type="checkbox" disabled="" /> One or more modules can be specified in one invocation of alppb
  • <input type="checkbox" disabled="" /> Allow specification of a requirements.txt file to use as a list of all modules to build
  • <input type="checkbox" disabled="" /> Specify download location of the artifact
  • <input type="checkbox" disabled="" /> Create an s3 bucket when an arg is specified
  • <input type="checkbox" disabled="" /> Allow user to optionally specify an IAM role
  • <input type="checkbox" disabled="" /> Specify the Python version that should be used to build the package (choices come from supported AWS Lambda versions)
  • <input type="checkbox" disabled="" /> Dockerize and submit to Dockerhub

FAQs

  1. Why AWS CodeBuild? Why not X instead?

AWS CodeBuild has a perpetual free tier and it's super easy to spin up, and teardown, a build job. Further, we can easily specify various Docker images to use for the build that match the AWS Lambda environment. I will likely add support for other build methods/services. If you have a suggestion, please open an issue or contact me on Twitter @irlrobot.

  1. What image is being used for CodeBuild? Can I inspect the image being used for the build?

There are three images, one for each version of Python supported by AWS Lambda:

Each image is running Amazon Linux 1 version 2017.03.1.20170812 which is what AWS Lambda uses.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for alppb, version 0.4.2
Filename, size File type Python version Upload date Hashes
Filename, size alppb-0.4.2-py2.py3-none-any.whl (11.4 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size alppb-0.4.2.tar.gz (10.9 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page