Skip to main content

User-friendly deploy and management tool for AWS Lambda function.

Project description

Lamvery
=======

|Build Status| |Coverage| |Scrutinizer Code Quality| |Version|

Description
===========

User-friendly deploy and management tool for AWS Lambda function.

Why user-friendly?
~~~~~~~~~~~~~~~~~~

The format of the configuration file is ``YAML``\ (with ``Jinja2``)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- ``YAML`` is user-friendly than ``JSON``
- We can avoid some redundant contents by ``Jinja2`` template engine

Additional features that are not in the standard Lambda functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Passing the environment variables
- Passing the confidential information use ``KMS`` encryption
- Passing the confidential files use ``KMS`` encryption
- And more

More useful features for deploying and invoking our functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- We can build, configure, deploy, rollback and invoke with a single
command
- We can rollback **correctly** to the previous version of the alias
- Deply(build) hooks
- And more

More useful features for using and managing the related services
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- API Gateway (Swagger)
- CloudWatch Events
- CloudWatch Logs
- And more

Requirements
============

- Python2.7
- pip

Recommends
==========

- virtualenv
**Automatically collect the lightweighted and compiled libraries in
the virtualenv environment.**

Installation
============

PyPI
----

.. code:: sh

pip install lamvery

Apt
---

.. code:: sh

echo "deb https://dl.bintray.com/willyworks/deb trusty main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install lamvery
export PATH=/opt/lamvery/bin:$PATH

Yum
---

.. code:: sh

echo "
[bintraybintray-willyworks-rpm]
name=bintray-willyworks-rpm
baseurl=https://dl.bintray.com/willyworks/rpm/centos/\$releaserver/\$basearch/
gpgcheck=0
enabled=1
" | sudo tee -a /etc/yum.repos.d/bintray-willyworks-rpm.repo
sudo yum install lamvery
export PATH=/opt/lamvery/bin:$PATH

Setup and configuration
=======================

First,

.. code:: sh

lamvery init

| And then edit your ``.lamvery.yml`` like so.
| The configuration is written in YAML syntax with ``jinja2`` template.
| Environment variables are stored in the ``env`` variable.

General settings (default: ``.lamvery.yml``)
--------------------------------------------

.. code:: yml

profile: private
region: us-east-1
versioning: true
default_alias: test
clean_build: false
configuration:
name: lamvery-test
runtime: python2.7
role: {{ env['AWS_LAMBDA_ROLE'] }}
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128
environment_variables:
MYENV: foobar
vpc_config:
subnets:
- subnet-cadf2993
security_groups:
- sg-4d095028

profile
~~~~~~~

The name of a profile to use. If not given, it depends on ``boto3``.

region
~~~~~~

| The region name of your environment.
| If you doesn't set this option, it depends on ``boto3``.

versioning
~~~~~~~~~~

Enable the function versioning.

default\_alias
~~~~~~~~~~~~~~

The alias when it has not been specified in the ``-a`` or ``--alias``
option.

clean\_build
~~~~~~~~~~~~

Build the archive(zip) in the temporary area.

configuration
~~~~~~~~~~~~~

- | name
| The name of your function.

- | runtime
| The runtime environment for the Lambda function you are uploading.
| Currently, ``lamvery`` supports ``python2.7`` and ``nodejs``.

- | role
| The Amazon Resource Name (ARN) of the IAM role for your function.

- | handler
| The function within your code that Lambda calls to begin execution.

- | description
| The description of your function.

- | timeout
| The function execution time(seconds) at which Lambda should
terminate the function.

- | memory\_size
| The amount of memory for your function environment.

- | environment\_variables
| Environment variables.

- vpc\_config
The VPC configurations for the function to access resources in your
VPC.
- subnets
The Subnet ids in your VPC.
- | security\_groups
| The SecurityGroup ids in your VPC.

CloudWatch Events settings (default: ``.lamvery.event.yml``)
------------------------------------------------------------

.. code:: yml

rules:
- name: foo
description: bar
schedule: 'rate(5 minutes)'
targets:
- id: test-target-id
input:
this:
- is: a
- sample: input

rules
~~~~~

CloudWatch Event Rules.

- | NAME
| The name of CloudWatch Event Rule.

- | description
| The description of CloudWatch Event Rule.

- | schedule
| The schedule expression of CloudWatch Event Rule.

- | disabled
| When this setting is true, change the state of CloudWatch Event
Rule to ``DISABLED``.
| default: ``false``

- targets
The targets of CloudWatch Event Rule.
- id
The unique target assignment ID.
- input
Arguments passed to the target.
- | input\_path
| The value of the JSONPath that is used for extracting part of the
matched event when passing it to the target.

*``input`` and ``input_path`` are mutually-exclusive and optional
parameters of a target.*

Secret informations (default: ``.lamvery.secret.yml``)
------------------------------------------------------

.. code:: yml

key_id: {{ env['AWS_KMS_KEY_ID'] }}
cipher_texts:
foo: CiC4xW9lg7HaxaueeN+d9yJMyY1uw1i7tYVvQz9I8+e2UBKXAQEBAgB4uMVvZYOx2sWrnnjfnfciTMmNbsNYu7WFb0M/SPPntlAAAABuMGwGCSqGSIb3DQEHBqBfMF0CAQAwWAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAzWTJWk/69T8NTBquoCARCAK2Hg2de71hzwjiMKkfMSG2G1Olj1EjxajS+3PsFVTPZ91Oi/AjR1aMqAI8U=

key\_id
~~~~~~~

The ID of your encryption key on KMS.

cipher\_texts
~~~~~~~~~~~~~

The name and cipher texts to pass to the lambda function.

secret\_files
~~~~~~~~~~~~~

The filename and the encrypted contents to pass to the lambda function.

Excluded patterns from the archive (default: ``.lamvery.exclude.yml``)
----------------------------------------------------------------------

.. code:: yml

- ^\.lamvery\.yml$
- ^\.lamvery\.event\.yml$
- ^\.lamvery\.secret\.yml$
- ^\.lamvery\.exclude\.yml$

Exclude files or directories using regular expression.

Action hooks (default: ``.lamvery.hook.yml``)
---------------------------------------------

.. code:: yml

build:
pre:
- pip install -r requirements.txt -t ./
post: []

build
~~~~~

The hooks for ``build``\ (and ``deploy``)

- | pre
| The commands that run before building.

- | post
| The commands that run after building.

API Gateway integration (default: ``.lamvery.api.yml``)
-------------------------------------------------------

.. code:: yml

api_id: myipugal74
stage: dev
cors:
origin: '*'
methods:
- GET
- OPTION
headers:
- Content-Type
- X-Amz-Date
- Authorization
- X-Api-Key
configuration:
swagger: '2.0'
info:
title: Sample API
schemes:
- https
paths:
/:
get:
produces:
- application/json
parameters:
- name: sample
in: query
required: false
type: string
responses:
'200':
description: 200 response
schema:
$ref: '#/definitions/Sample'
definitions:
Sample:
type: object

api\_id
~~~~~~~

| The id of your REST API.
| This is written on automatically when you deployed your API with the
``-w`` option.

stage
~~~~~

The name of the stage in API Gateway.

cors
~~~~

| CORS (Cross-Origin Resource Sharing) options.
| If you did not set the ``x-amazon-apigateway-integration`` option,
these are set automatically.

- origin
For the response's header that named "Access-Control-Allow-Origin".
- headers
For the response's header that named "Access-Control-Allow-Headers".
- methods
For the response's header that named "Access-Control-Allow-Methods".

configuration
~~~~~~~~~~~~~

| The settings of your APIs written in the ``Swagger`` format.
| If you did not set the ``x-amazon-apigateway-integration`` option,
these are set automatically.

Commands
========

build
~~~~~

- Build and archive your code and libraries to
``<your-function-name>.zip``
- Store the secret informations to the archive

.. code:: sh

lamvery build [-e <env-name>=<env-value>]

deploy
~~~~~~

- Build and deploy your code and libraries
- Store the secret informations to the archive
- Update configuration of the function
- Set alias to a version of the function

.. code:: sh

lamvery deploy [-e <env-name>=<env-value>] [-a <alias>]

rollback
~~~~~~~~

- Rollback to the previous version of the function

**You must do one of the following to use this command.**

- Deploy with ``publish(-p,--publish)`` and ``alias(-a, --alias)``
options.
- Turn on(true) ``versioning`` and set a value to ``default_alias`` in
the configuration file.

::

lamvery rollback [-a <alias>]

set-alias
~~~~~~~~~

- Set alias to a version of the function

.. code:: sh

lamvery set-alias -a <alias> -v <alias-version>

encrypt
~~~~~~~

- Encrypt a text value using KMS

.. code:: sh

lamvery encrypt [-s] -n <secret-name> <secret-value>

encrypt-file
~~~~~~~~~~~~

- Encrypt a file using KMS

.. code:: sh

lamvery encrypt [-s] -n <filename> <local-file-path>

decrypt
~~~~~~~

- Decrypt the secret information using KMS

.. code:: sh

lamvery decrypt -n <secret-name>

events
~~~~~~

- Apply CloudWatch Events settings

.. code:: sh

lamvery events [-k] [-a <alias>]

invoke
~~~~~~

- Invoke the function and output result

.. code:: sh

lamvery invoke [-a <alias>] [-v <version>] '{"foo": "bar"}'

or

.. code:: sh

lamvery invoke [-a <alias>] [-v <version>] path/to/input.json

logs
~~~~

- Watch the log events of the function

.. code:: sh

lamvery logs [-f] [-F <filter>] [-s <start-time-string>] [-i <interval-seconds>]

api
~~~

- Manage your APIs

.. code:: sh

lamvery api [-n] [-r] [-s <stage-name>] [-w]

generate
~~~~~~~~

- Generate skeleton files

.. code:: sh

lamvery generate -k <kind-of-file>

Options
-------

``-a`` or ``--alias``
~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
``deploy``,\ ``set-alias``,\ ``invoke``,\ ``rollback``,\ ``events``,\ ``api``
commands.
| Alias for a version of the function.

``-c`` or ``--conf-file``
~~~~~~~~~~~~~~~~~~~~~~~~~

| This option needed by all commands.
| Specify the configuration file.
| default: ``.lamvery.yml``

``-d`` or ``--dry-run``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
``deploy``,\ ``alias``,\ ``rollback``,\ ``events``,\ ``api`` commands.
| Output the difference of configuration and the alias without updating.

``-s`` or ``--single-file``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive only the lambda function file, so you can inline edit in the
AWS Management Console.

``-l`` or ``--no-libs``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive without all libraries.

``-n`` or ``--secret-name``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``encrypt`` and ``decrypt`` commands.
| The name of the secret value.

``-p`` or ``--publish``
~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``deploy`` command. Publish the
version as an atomic operation.

``-k`` or ``--keep-empty-events``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``events`` command. Keep the empty
CloudWatch Event Rule that does not have CloudWatch Event Target.

``-s`` or ``--store``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt`` command.
| Store encripted value to configuration file (default:
``.lamvery.yml``).
| Requires the ``-n`` or ``--secret-name`` option.

``-v`` or ``--version``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``set-alias``,\ ``invoke``,\ ``rollback``
commands.
| Version of the function.

``-f`` or ``--follow``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Watch the log events and updates the display (like ``tail -f``).

``-F`` or ``--filter``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Filtering pattern for the log messages.

``-i`` or ``--interval``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Intervals(seconds) to watch the log events.

``-s`` or ``--start``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Time to start the log events watching.
| Examples: ``yesterday``,\ ``"-1 h"``, ``"2016-01-01"``,
``"2016-01-01 10:20:30"``

``-t`` or ``--target``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``set-alias`` command.
| The alias of the version that is targeted for setting alias.

``-e`` or ``--env``
~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` commands.
| Environment variables that pass to the function.
| **This option can be used repeatedly to pass multiple variables.**
| Examples: ``FOO=BAR``

``-r`` or ``--remove``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Remove your APIs.

``-s`` or ``--stage``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| The name of the stage in API Gateway.

``-w`` or ``--write-id``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Write the id of your API to the configuration file (default:
``.lamvery.api.yml``)

``-n`` or ``--no-integrate``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Without automatic integration.

``-k`` or ``--kind``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``generate`` command.
| The kind of the file.
| Allowed values: ``function``

``-n`` or ``--name``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt-file`` command.
| The name of the secret file.

How to use the confidential information in the lambda function
==============================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
]
}
]
}

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

- .lamvery.yml

.. code:: yml

profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
name: sample_lambda_function
runtime: python2.7 # or nodejs
role: arn:aws:iam::000000000000:role/lambda_basic_execution
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128

- .lamvery.secret.yml

.. code:: yml

key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential information to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery encrypt -s -n foo "This is a secret"

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

lamvery generate -k function

Code example:

- Python

.. code:: py

import lamvery

def lambda_handler(event, context):
print(lamvery.secret.get('foo'))

- Node.js

.. code:: js

var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
lamvery.secret.get('foo', function(err, data) {
console.log(data);
});
}

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery invoke {}

Result example:

::

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

How to use the confidential file in the lambda function
=======================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
]
}
]
}

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

- .lamvery.yml

.. code:: yml

profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
name: sample_lambda_function
runtime: python2.7 # or nodejs
role: arn:aws:iam::000000000000:role/lambda_basic_execution
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128

- .lamvery.secret.yml

.. code:: yml

key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential file to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery encrypt-file -s -n foo.txt /path/to/local/confidential/file

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

lamvery generate -k function

Code example:

- Python

.. code:: py

import lamvery

def lambda_handler(event, context):
print(open(lamvery.secret.file('foo.txt'), 'r').read())

- Node.js

.. code:: js

var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
lamvery.secret.file('foo.txt', function(err, path) {
fs.readFile(path, 'utf-8', function(err, txt) {
console.log(txt);
});
});
}

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery invoke {}

Result example:

::

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret file
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

Development
-----------

- Source hosted at `GitHub <https://github.com/marcy-terui/lamvery>`__
- Report issues/questions/feature requests on `GitHub
Issues <https://github.com/marcy-terui/lamvery/issues>`__

Pull requests are very welcome! Make sure your patches are well tested.
Ideally create a topic branch for every separate change you make. For
example:

1. Fork the repo
2. Create your feature branch (``git checkout -b my-new-feature``)
3. Commit your changes (``git commit -am 'Added some feature'``)
4. Push to the branch (``git push origin my-new-feature``)
5. Create new Pull Request

Authors
-------

Created and maintained by `Masashi
Terui <https://github.com/marcy-terui>`__ (marcy9114@gmail.com)

License
-------

MIT License (see
`LICENSE <https://github.com/marcy-terui/lamvery/blob/master/LICENSE>`__)

.. |Build Status| image:: https://img.shields.io/travis/marcy-terui/lamvery/master.svg
:target: http://travis-ci.org/marcy-terui/lamvery
.. |Coverage| image:: https://img.shields.io/coveralls/marcy-terui/lamvery.svg
:target: https://coveralls.io/github/marcy-terui/lamvery
.. |Scrutinizer Code Quality| image:: https://scrutinizer-ci.com/g/marcy-terui/lamvery/badges/quality-score.png?b=master
:target: https://scrutinizer-ci.com/g/marcy-terui/lamvery/?branch=master
.. |Version| image:: https://img.shields.io/pypi/v/lamvery.svg
:target: https://pypi.python.org/pypi/lamvery

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lamvery-0.18.2.tar.gz (32.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page