Skip to main content
Join the official Python Developers Survey 2018 and win valuable prizes: Start the survey!

Run Any Python 3 version on AWS Lambda

Project description

# Disclaimer
* theScore does not use this in production. This library was built when AWS Lambda was stuck at 2.7 for a long time and we wanted to use Python 3. Two days after this idea succeeded, AWS announced support for 3.6 :). However, this may still be of use to those who want to use versions other than the official version

# Marol

Run any Python 3 version instead of the standard AWS Lambda Python version.

## Python 3 version support

* Tested on 3.6.1, 3.6.2, 3.6.3


## Instructions
You pass in your handler file(e.g. `my_handler.py`) and it returns back:

* `handler_python3.py`. `my_handler.py` is renamed to `handler_python3.py` and code is added to make it callable from a python2 handler
* `handler.py`. This is the Python2 module that is called which delegates to the Python3 handler file
* path to `marol_venv` folder. This is the folder which contains all the executables for the Python version that you chose.

To use Marol, your `my_handler.py` will have the following constraints:

* The handler function should be named `handler`
* The file cannot be called `handler.py`(This is reserved for the Python2 handler which will execute the Python3 handler
* There should be no `main` statement in the handler file
* One should not have functions with the following names:
1. `load`
2. `read_execution_uuid`


## Installation
### Pre-requisites
* Ensure `docker` is installed and running on the machine where Marol is executed.

### Mac OS X
* Open the terminal and run `/Applications/<Python Version>/Install Certificates.command`


## Usage

Your deployment scripts for lambda should specify `handler.py` and `handler` as the starting point for your Lambda function. This is the Python2 handler returned by Marol which calls your modified Python3 handler.


```python
import marol
my_project_site_packages = ...
my_deploy_lambda_function(lambda_name,
marol.get_lambda_files('~/mypath/mypy3handler.py', '3.6.1') + my_project_site_packages,
'lambda_s3_exec_role',
128,
300,
lambda_description='Test for Python 3',
handler_name="handler",
handler_file="handler.py",
region_name='us-east-1')

```

`get_lambda_files` will check `MAROL_HOME` if `marol_venv` exists for the particular version that you want. If it exists, you will get back that path. If it does not exist,`marol` will
* download the python source
* build the binaries using a [docker image](https://github.com/lambci/docker-lambda) which mirrors the AWS Lambda environment
* create a basic `marol_venv` and store it at `MAROL_HOME/staging/<python_version>`

## Future Work
* Some of the context attributes are not supported yet.
* Ensure that the context object is updated for future changes

## Notes
* This adds an additional 2-5 seconds to execution time.
* This only provides the core Python libraries.
* Default Marol home is `<HOME>/.marol`.
* It will create `marol_venv` in `<HOME>/.marol/staging/<python_version>`


## Notes for those thinking about security
* This uses an unofficial [docker image](https://github.com/lambci/docker-lambda)

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
marol-0.0.4.tar.gz (5.4 kB) Copy SHA256 hash SHA256 Source None Oct 30, 2017

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page