Skip to main content

Run Any Python 3 version on AWS Lambda

Project description

# Disclaimer
* theScore does not use this in production. This library was built when AWS Lambda was stuck at 2.7 for a long time and we wanted to use Python 3. Two days after this idea succeeded, AWS announced support for 3.6 :). However, this may still be of use to those who want to use versions other than the official version

# Marol

Run any Python 3 version instead of the standard AWS Lambda Python version.

## Python 3 version support

* Tested on 3.6.1, 3.6.2, 3.6.3


## Instructions
You pass in your handler file(e.g. `my_handler.py`) and it returns back:

* `handler_python3.py`. `my_handler.py` is renamed to `handler_python3.py` and code is added to make it callable from a python2 handler
* `handler.py`. This is the Python2 module that is called which delegates to the Python3 handler file
* path to `marol_venv` folder. This is the folder which contains all the executables for the Python version that you chose.

To use Marol, your `my_handler.py` will have the following constraints:

* The handler function should be named `handler`
* The file cannot be called `handler.py`(This is reserved for the Python2 handler which will execute the Python3 handler
* There should be no `main` statement in the handler file
* One should not have functions with the following names:
1. `load`
2. `read_execution_uuid`


## Installation
### Pre-requisites
* Ensure `docker` is installed and running on the machine where Marol is executed.

### Mac OS X
* Open the terminal and run `/Applications/<Python Version>/Install Certificates.command`


## Usage

Your deployment scripts for lambda should specify `handler.py` and `handler` as the starting point for your Lambda function. This is the Python2 handler returned by Marol which calls your modified Python3 handler.


```python
import marol
my_project_site_packages = ...
my_deploy_lambda_function(lambda_name,
marol.get_lambda_files('~/mypath/mypy3handler.py', '3.6.1') + my_project_site_packages,
'lambda_s3_exec_role',
128,
300,
lambda_description='Test for Python 3',
handler_name="handler",
handler_file="handler.py",
region_name='us-east-1')

```

`get_lambda_files` will check `MAROL_HOME` if `marol_venv` exists for the particular version that you want. If it exists, you will get back that path. If it does not exist,`marol` will
* download the python source
* build the binaries using a [docker image](https://github.com/lambci/docker-lambda) which mirrors the AWS Lambda environment
* create a basic `marol_venv` and store it at `MAROL_HOME/staging/<python_version>`

## Future Work
* Some of the context attributes are not supported yet.
* Ensure that the context object is updated for future changes

## Notes
* This adds an additional 2-5 seconds to execution time.
* This only provides the core Python libraries.
* Default Marol home is `<HOME>/.marol`.
* It will create `marol_venv` in `<HOME>/.marol/staging/<python_version>`


## Notes for those thinking about security
* This uses an unofficial [docker image](https://github.com/lambci/docker-lambda)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

marol-0.0.4.tar.gz (5.4 kB view details)

Uploaded Source

File details

Details for the file marol-0.0.4.tar.gz.

File metadata

  • Download URL: marol-0.0.4.tar.gz
  • Upload date:
  • Size: 5.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for marol-0.0.4.tar.gz
Algorithm Hash digest
SHA256 6bd4674d7b462d4cf4335c623258c759ac2553f71ad0c90ef60881d1e2bf6d81
MD5 1e12ec98a64fe1093c3a2e87e07f3513
BLAKE2b-256 2ac3adda6a2297134fad9d766c50fcb51737a6f9dcc29f75e077b68a4d7fa449

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page