lamvery 0.18.2

User-friendly deploy and management tool for AWS Lambda function.

Lamvery
=======

|Build Status| |Coverage| |Scrutinizer Code Quality| |Version|

Description
===========

User-friendly deploy and management tool for AWS Lambda function.

Why user-friendly?
~~~~~~~~~~~~~~~~~~

The format of the configuration file is ``YAML``\ (with ``Jinja2``)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- ``YAML`` is user-friendly than ``JSON``
- We can avoid some redundant contents by ``Jinja2`` template engine

Additional features that are not in the standard Lambda functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Passing the environment variables
- Passing the confidential information use ``KMS`` encryption
- Passing the confidential files use ``KMS`` encryption
- And more

More useful features for deploying and invoking our functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- We can build, configure, deploy, rollback and invoke with a single
command
- We can rollback **correctly** to the previous version of the alias
- Deply(build) hooks
- And more

More useful features for using and managing the related services
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- API Gateway (Swagger)
- CloudWatch Events
- CloudWatch Logs
- And more

Requirements
============

- Python2.7
- pip

Recommends
==========

- virtualenv
**Automatically collect the lightweighted and compiled libraries in
the virtualenv environment.**

Installation
============

PyPI
----

.. code:: sh

pip install lamvery

Apt
---

.. code:: sh

echo "deb https://dl.bintray.com/willyworks/deb trusty main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install lamvery
export PATH=/opt/lamvery/bin:$PATH

Yum
---

.. code:: sh

echo "
[bintraybintray-willyworks-rpm]
name=bintray-willyworks-rpm
baseurl=https://dl.bintray.com/willyworks/rpm/centos/\$releaserver/\$basearch/
gpgcheck=0
enabled=1
" | sudo tee -a /etc/yum.repos.d/bintray-willyworks-rpm.repo
sudo yum install lamvery
export PATH=/opt/lamvery/bin:$PATH

Setup and configuration
=======================

First,

.. code:: sh

lamvery init

| And then edit your ``.lamvery.yml`` like so.
| The configuration is written in YAML syntax with ``jinja2`` template.
| Environment variables are stored in the ``env`` variable.

General settings (default: ``.lamvery.yml``)
--------------------------------------------

.. code:: yml

profile: private
region: us-east-1
versioning: true
default_alias: test
clean_build: false
configuration:
name: lamvery-test
runtime: python2.7
role: {{ env['AWS_LAMBDA_ROLE'] }}
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128
environment_variables:
MYENV: foobar
vpc_config:
subnets:
- subnet-cadf2993
security_groups:
- sg-4d095028

profile
~~~~~~~

The name of a profile to use. If not given, it depends on ``boto3``.

region
~~~~~~

| The region name of your environment.
| If you doesn't set this option, it depends on ``boto3``.

versioning
~~~~~~~~~~

Enable the function versioning.

default\_alias
~~~~~~~~~~~~~~

The alias when it has not been specified in the ``-a`` or ``--alias``
option.

clean\_build
~~~~~~~~~~~~

Build the archive(zip) in the temporary area.

configuration
~~~~~~~~~~~~~

- | name
| The name of your function.

- | runtime
| The runtime environment for the Lambda function you are uploading.
| Currently, ``lamvery`` supports ``python2.7`` and ``nodejs``.

- | role
| The Amazon Resource Name (ARN) of the IAM role for your function.

- | handler
| The function within your code that Lambda calls to begin execution.

- | description
| The description of your function.

- | timeout
| The function execution time(seconds) at which Lambda should
terminate the function.

- | memory\_size
| The amount of memory for your function environment.

- | environment\_variables
| Environment variables.

- vpc\_config
The VPC configurations for the function to access resources in your
VPC.
- subnets
The Subnet ids in your VPC.
- | security\_groups
| The SecurityGroup ids in your VPC.

CloudWatch Events settings (default: ``.lamvery.event.yml``)
------------------------------------------------------------

.. code:: yml

rules:
- name: foo
description: bar
schedule: 'rate(5 minutes)'
targets:
- id: test-target-id
input:
this:
- is: a
- sample: input

rules
~~~~~

CloudWatch Event Rules.

- | NAME
| The name of CloudWatch Event Rule.

- | description
| The description of CloudWatch Event Rule.

- | schedule
| The schedule expression of CloudWatch Event Rule.

- | disabled
| When this setting is true, change the state of CloudWatch Event
Rule to ``DISABLED``.
| default: ``false``

- targets
The targets of CloudWatch Event Rule.
- id
The unique target assignment ID.
- input
Arguments passed to the target.
- | input\_path
| The value of the JSONPath that is used for extracting part of the
matched event when passing it to the target.

*``input`` and ``input_path`` are mutually-exclusive and optional
parameters of a target.*

Secret informations (default: ``.lamvery.secret.yml``)
------------------------------------------------------

.. code:: yml

key_id: {{ env['AWS_KMS_KEY_ID'] }}
cipher_texts:
foo: CiC4xW9lg7HaxaueeN+d9yJMyY1uw1i7tYVvQz9I8+e2UBKXAQEBAgB4uMVvZYOx2sWrnnjfnfciTMmNbsNYu7WFb0M/SPPntlAAAABuMGwGCSqGSIb3DQEHBqBfMF0CAQAwWAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAzWTJWk/69T8NTBquoCARCAK2Hg2de71hzwjiMKkfMSG2G1Olj1EjxajS+3PsFVTPZ91Oi/AjR1aMqAI8U=

key\_id
~~~~~~~

The ID of your encryption key on KMS.

cipher\_texts
~~~~~~~~~~~~~

The name and cipher texts to pass to the lambda function.

secret\_files
~~~~~~~~~~~~~

The filename and the encrypted contents to pass to the lambda function.

Excluded patterns from the archive (default: ``.lamvery.exclude.yml``)
----------------------------------------------------------------------

.. code:: yml

- ^\.lamvery\.yml$
- ^\.lamvery\.event\.yml$
- ^\.lamvery\.secret\.yml$
- ^\.lamvery\.exclude\.yml$

Exclude files or directories using regular expression.

Action hooks (default: ``.lamvery.hook.yml``)
---------------------------------------------

.. code:: yml

build:
pre:
- pip install -r requirements.txt -t ./
post: []

build
~~~~~

The hooks for ``build``\ (and ``deploy``)

- | pre
| The commands that run before building.

- | post
| The commands that run after building.

API Gateway integration (default: ``.lamvery.api.yml``)
-------------------------------------------------------

.. code:: yml

api_id: myipugal74
stage: dev
cors:
origin: '*'
methods:
- GET
- OPTION
headers:
- Content-Type
- X-Amz-Date
- Authorization
- X-Api-Key
configuration:
swagger: '2.0'
info:
title: Sample API
schemes:
- https
paths:
/:
get:
produces:
- application/json
parameters:
- name: sample
in: query
required: false
type: string
responses:
'200':
description: 200 response
schema:
$ref: '#/definitions/Sample'
definitions:
Sample:
type: object

api\_id
~~~~~~~

| The id of your REST API.
| This is written on automatically when you deployed your API with the
``-w`` option.

stage
~~~~~

The name of the stage in API Gateway.

cors
~~~~

| CORS (Cross-Origin Resource Sharing) options.
| If you did not set the ``x-amazon-apigateway-integration`` option,
these are set automatically.

- origin
For the response's header that named "Access-Control-Allow-Origin".
- headers
For the response's header that named "Access-Control-Allow-Headers".
- methods
For the response's header that named "Access-Control-Allow-Methods".

configuration
~~~~~~~~~~~~~

| The settings of your APIs written in the ``Swagger`` format.
| If you did not set the ``x-amazon-apigateway-integration`` option,
these are set automatically.

Commands
========

build
~~~~~

- Build and archive your code and libraries to
``<your-function-name>.zip``
- Store the secret informations to the archive

.. code:: sh

lamvery build [-e <env-name>=<env-value>]

deploy
~~~~~~

- Build and deploy your code and libraries
- Store the secret informations to the archive
- Update configuration of the function
- Set alias to a version of the function

.. code:: sh

lamvery deploy [-e <env-name>=<env-value>] [-a <alias>]

rollback
~~~~~~~~

- Rollback to the previous version of the function

**You must do one of the following to use this command.**

- Deploy with ``publish(-p,--publish)`` and ``alias(-a, --alias)``
options.
- Turn on(true) ``versioning`` and set a value to ``default_alias`` in
the configuration file.

::

lamvery rollback [-a <alias>]

set-alias
~~~~~~~~~

- Set alias to a version of the function

.. code:: sh

lamvery set-alias -a <alias> -v <alias-version>

encrypt
~~~~~~~

- Encrypt a text value using KMS

.. code:: sh

lamvery encrypt [-s] -n <secret-name> <secret-value>

encrypt-file
~~~~~~~~~~~~

- Encrypt a file using KMS

.. code:: sh

lamvery encrypt [-s] -n <filename> <local-file-path>

decrypt
~~~~~~~

- Decrypt the secret information using KMS

.. code:: sh

lamvery decrypt -n <secret-name>

events
~~~~~~

- Apply CloudWatch Events settings

.. code:: sh

lamvery events [-k] [-a <alias>]

invoke
~~~~~~

- Invoke the function and output result

.. code:: sh

lamvery invoke [-a <alias>] [-v <version>] '{"foo": "bar"}'

or

.. code:: sh

lamvery invoke [-a <alias>] [-v <version>] path/to/input.json

logs
~~~~

- Watch the log events of the function

.. code:: sh

lamvery logs [-f] [-F <filter>] [-s <start-time-string>] [-i <interval-seconds>]

api
~~~

- Manage your APIs

.. code:: sh

lamvery api [-n] [-r] [-s <stage-name>] [-w]

generate
~~~~~~~~

- Generate skeleton files

.. code:: sh

lamvery generate -k <kind-of-file>

Options
-------

``-a`` or ``--alias``
~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
``deploy``,\ ``set-alias``,\ ``invoke``,\ ``rollback``,\ ``events``,\ ``api``
commands.
| Alias for a version of the function.

``-c`` or ``--conf-file``
~~~~~~~~~~~~~~~~~~~~~~~~~

| This option needed by all commands.
| Specify the configuration file.
| default: ``.lamvery.yml``

``-d`` or ``--dry-run``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
``deploy``,\ ``alias``,\ ``rollback``,\ ``events``,\ ``api`` commands.
| Output the difference of configuration and the alias without updating.

``-s`` or ``--single-file``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive only the lambda function file, so you can inline edit in the
AWS Management Console.

``-l`` or ``--no-libs``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive without all libraries.

``-n`` or ``--secret-name``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``encrypt`` and ``decrypt`` commands.
| The name of the secret value.

``-p`` or ``--publish``
~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``deploy`` command. Publish the
version as an atomic operation.

``-k`` or ``--keep-empty-events``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``events`` command. Keep the empty
CloudWatch Event Rule that does not have CloudWatch Event Target.

``-s`` or ``--store``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt`` command.
| Store encripted value to configuration file (default:
``.lamvery.yml``).
| Requires the ``-n`` or ``--secret-name`` option.

``-v`` or ``--version``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``set-alias``,\ ``invoke``,\ ``rollback``
commands.
| Version of the function.

``-f`` or ``--follow``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Watch the log events and updates the display (like ``tail -f``).

``-F`` or ``--filter``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Filtering pattern for the log messages.

``-i`` or ``--interval``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Intervals(seconds) to watch the log events.

``-s`` or ``--start``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Time to start the log events watching.
| Examples: ``yesterday``,\ ``"-1 h"``, ``"2016-01-01"``,
``"2016-01-01 10:20:30"``

``-t`` or ``--target``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``set-alias`` command.
| The alias of the version that is targeted for setting alias.

``-e`` or ``--env``
~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` commands.
| Environment variables that pass to the function.
| **This option can be used repeatedly to pass multiple variables.**
| Examples: ``FOO=BAR``

``-r`` or ``--remove``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Remove your APIs.

``-s`` or ``--stage``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| The name of the stage in API Gateway.

``-w`` or ``--write-id``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Write the id of your API to the configuration file (default:
``.lamvery.api.yml``)

``-n`` or ``--no-integrate``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Without automatic integration.

``-k`` or ``--kind``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``generate`` command.
| The kind of the file.
| Allowed values: ``function``

``-n`` or ``--name``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt-file`` command.
| The name of the secret file.

How to use the confidential information in the lambda function
==============================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
]
}
]
}

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

- .lamvery.yml

.. code:: yml

profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
name: sample_lambda_function
runtime: python2.7 # or nodejs
role: arn:aws:iam::000000000000:role/lambda_basic_execution
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128

- .lamvery.secret.yml

.. code:: yml

key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential information to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery encrypt -s -n foo "This is a secret"

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

lamvery generate -k function

Code example:

- Python

.. code:: py

import lamvery

def lambda_handler(event, context):
print(lamvery.secret.get('foo'))

- Node.js

.. code:: js

var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
lamvery.secret.get('foo', function(err, data) {
console.log(data);
});
}

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery invoke {}

Result example:

::

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

How to use the confidential file in the lambda function
=======================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
]
}
]
}

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

- .lamvery.yml

.. code:: yml

profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
name: sample_lambda_function
runtime: python2.7 # or nodejs
role: arn:aws:iam::000000000000:role/lambda_basic_execution
handler: lambda_function.lambda_handler
description: This is sample lambda function.
timeout: 10
memory_size: 128

- .lamvery.secret.yml

.. code:: yml

key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential file to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery encrypt-file -s -n foo.txt /path/to/local/confidential/file

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

lamvery generate -k function

Code example:

- Python

.. code:: py

import lamvery

def lambda_handler(event, context):
print(open(lamvery.secret.file('foo.txt'), 'r').read())

- Node.js

.. code:: js

var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
lamvery.secret.file('foo.txt', function(err, path) {
fs.readFile(path, 'utf-8', function(err, txt) {
console.log(txt);
});
});
}

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

lamvery invoke {}

Result example:

::

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret file
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

Development
-----------

- Source hosted at `GitHub <https://github.com/marcy-terui/lamvery>`__
- Report issues/questions/feature requests on `GitHub
Issues <https://github.com/marcy-terui/lamvery/issues>`__

Pull requests are very welcome! Make sure your patches are well tested.
Ideally create a topic branch for every separate change you make. For
example:

1. Fork the repo
2. Create your feature branch (``git checkout -b my-new-feature``)
3. Commit your changes (``git commit -am 'Added some feature'``)
4. Push to the branch (``git push origin my-new-feature``)
5. Create new Pull Request

Authors
-------

Created and maintained by `Masashi
Terui <https://github.com/marcy-terui>`__ ([e-mail])

License
-------

MIT License (see
`LICENSE <https://github.com/marcy-terui/lamvery/blob/master/LICENSE>`__)

.. |Build Status| image:: https://img.shields.io/travis/marcy-terui/lamvery/master.svg
:target: http://travis-ci.org/marcy-terui/lamvery
.. |Coverage| image:: https://img.shields.io/coveralls/marcy-terui/lamvery.svg
:target: https://coveralls.io/github/marcy-terui/lamvery
.. |Scrutinizer Code Quality| image:: https://scrutinizer-ci.com/g/marcy-terui/lamvery/badges/quality-score.png?b=master
:target: https://scrutinizer-ci.com/g/marcy-terui/lamvery/?branch=master
.. |Version| image:: https://img.shields.io/pypi/v/lamvery.svg
:target: https://pypi.python.org/pypi/lamvery