humilis 0.0.5

Helps you deploy infrastructure in the AWS cloud

humilis

Helps you deploy AWS infrastructure with Cloudformation.

This project is originally based on the cumulus. project. See CUMULUS_LICENSE for license information.

Installation

To install the latest “stable” version:

pip install humilis

To install the development version:

pip install git+https://github.com/germangh/humilis

Development environment

Assuming you have virtualenv installed:

make develop

. .env/bin/activate

Testing

At the moment, most tests are integration tests with the AWS SDK. This means that you will need to set up your system to access AWS resources if you want to run the test suite.

py.test tests

Quickstart

Define your infrastructure environment following the examples in the examples directory. Then to create the environment:

humilis create example-environment.yml

And to delete it:

humilis delete example-environment.yml

For now you can’t use humilis to update existing environments.

Humilis environments

A humilis environment is just a collection of cloudformation stacks that are required for an application. Instead of having a monolytic CF template for your complete application, humilis allows you to define infrastructure layers that are combined into an environment. Each humilis layer translates exactly into one CF template (therefore into one CF stack after the layer is deployed).

Breaking a complex infrastructure environment into smaller layers has at least two obvious advantages:

• Easier to maintain. It’s easier to maintain a simple layer that contains just a bunch of CF resources than serve a well-defined purpose.

• Easier to reuse. You should strive to define your infrastructure layers in such a way that you can reuse them across various environments. For instance, many projects may require a base layer that defines a VPC, a few subnets, a gateway and some routing tables, and maybe a (managed) NAT. You can define a humilis layer with those resources and have a set of layer parameters (e.g. the VPC CIDR) that will allow you to easily reuse it across environments.

Environment anatomy

An environment definition file is a yaml document that specifies the list of layers that form your enviroment. The file should be named as your environment. That is, for environment my-app-environment the environment description file should be called my-app-environment.yaml. The contents of the environment definition should be organized as follows:

---
my-app-environment:
    description:
        A description of what this environment is for
    layers:
        # The layers that you environment requires. They will be deployed in the
        # same order as you list them. Note that you can also pass parameters
        # to a layer (more on that later).
        - {layer: name_of_first_layer, layer_param: layer_value}
        - {layer: name_of_second_layer}
        - {layer: name_of_third_layer}

Layer anatomy

Anything associated to a given layer must be stored in a directory with the same name as the layer, within the same directory where the environment definition file is located. If we consider the my-app-environment environment we used above then your directory tree should look like this:

.
├── my-app-environment.yaml
├── name_of_first_layer
│   ├── meta.yaml
│   └── resources.yaml
├── name_of_second_layer
│   ├── meta.json
│   └── meta.yaml
└── name_of_third_layer
    ├── resources.json.j2
    └── resources.yaml.j2

A layer must contain at least two files:

• meta.yaml: Meta information about the layer such as a description, dependencies with other layers, and layer parameters.

• resources.yaml: Basically a CF template with the resources that the layer contains.

Those two files can also be in .json format (meta.json and resources.json). Or you can add the extension .j2 if you want the files to be pre-processed with the Jinja2 template compiler.

Below an example of how a layer meta.yaml may look like:

---
meta:
    description:
        Creates a VPC, that's it
    parameters:
        vpc_cidr:
            description: The CIDR block of the VPC
            value: 10.0.0.0/16

Above we declare only one layer parameter: vpc_cidr. humilis will make pass that parameter to Jinja2 when compiling any template contained in the layer. So the resources.yaml.j2 for that same layer may look like this:

---
resources:
    VPC:
        Type: "AWS::EC2::VPC"
        Properties:
            CidrBlock: {{ vpc_cidr }}

References

You can use references in your meta.yaml files to refer to thing other than resources within the same layer (to refer to resources within a layer you can simply use Cloudformation’s Ref or GetAtt functions). Humilis references are used by setting the value of a layer parameter to a dict that has a ref key. Below an a meta.yaml that refers to a resource (with a logical name VPC) that is contained in another layer (called vpc_layer):

---
meta:
    description:
        Creates an EC2 instance in the vpc created by the vpc layer
    dependencies:
        - vpc
    parameters:
        vpc:
            description: Physical ID of the VPC where the instance will be created
            value:
                ref:
                    parser: layer
                    parameters:
                        layer_name: vpc_layer
                        resource_name: VPC

Every reference must have a parser key that identifies the parser that should be used to parse the reference. The optional key parameters allows you to pass parameters to the reference parser. You can pass either named parameters (as a dict) or positional arguments (as a list). More information on reference parsers below.

Available reference parsers

TBD