nameless-deploy-tools 1.320

Tools for deploying to AWS via CloudFormation and Serverless framework that support a pull request based workflow

Nameless Deploy Tools

Released version 1.320

Nameless deploy tools are a set of tools to implement a true Infrastructure As Code workflow with various cloud infrastructure management tools. Currently supported tools are CloudFormation, AWS CDK, Serverless Framework, Terraform, Azure Resource Manager (with a YAML syntax) and Bicep.

Why Nameless?

A common analogy for cloud infrastructure has been to move from having pets with names that need lots of looking after, to cattle that has at most id's. It's time to move to the industrial age from the agrarian era. The infrastructure our applications runs now comes and goes, and we know at most some statistical information about the actual executions. Run times, memory usage, used bandwidth and the like. We no longer know even the id's of the things that actually run the code. Hence - nameless.

Rationale

We at Nitor are software engineers with mostly a developer or architect background, but a lot of us have had to work closely with various Operations teams around the world. DevOps has a natural appeal to us and immediately "infrastructure as code" meant for us that we should apply the best development practices to infrastructure development. It starts with version control and continues with testing new features in isolation and a workflow that supports this. Our teams usually take into use a feature branch workflow if it is feasible, and we expect all the tools and practices to support this. For infrastructure this type of branching means that you should be able to spin up enough of the infrastructure to be able to verify the changes you want to implement in production. Also, the testing environment should be close enough to the target environment for the results to be valid. So the differences between testing and production environments should be minimized and reviewable.

With the popular tools like Ansible, Terraform, Chef etc. you need to come up with and implement the ways to achieve the goals above. As far as I know, no tool besides ndt has at its core a thought-out way of a branching infrastructure development model.

What it is

nameless-deploy-tools works by defining Amazon Machine Images, Docker containers, Serverless services and deploying CloudFormation stacks of resources. CloudFormation stacks can also be defined with AWS CDK applications. All of the above can also be deployed using Terraform.

Installation

Requires Python 3.8 or newer.

pip install nameless-deploy-tools

Or in an isolated environment using pipx:

pipx install nameless-deploy-tools
pipx upgrade nameless-deploy-tools

Getting started

To use nameless-deploy-tools you need to set up a project repository that describes the images you want to build, and the stacks you want to deploy them in. See ndt-project-template for an example.

Here are few commands you can use. All of these are run in your project repository root. You need to have AWS credentials for command line access set up.

• To bake a new version of an image: ndt bake-image <image-name>
• To build a new Docker container image ndt bake-docker <component> <docker-name>
• To deploy a stack:
  • with a known AMI id: ndt deploy-stack <image-name> <stack-name> <AMI-id>
  • with the newest AMI id by a given bake job: ndt deploy-stack <image-name> <stack-name> "" <bake-job-name>
• To undeploy a stack: ndt undeploy-stack <image-name> <stack-name>

For full list of commands see here

Faster shell complete

You can additionally use a faster register-complete by running ./faster_register_complete.sh. This compiles C++ programs from the files n_utils/nameless-dt-register-complete.cpp and n_utils/nameless-dt-print-aws-profiles.cpp, and replaces the Python versions of nameless-dt-register-complete and nameless-dt-print-aws-profiles with these much faster compiled binaries.

Documentation

• Command Reference
• ndt workspace tooling
• Template Pre-Processing
• Multifactor Authentication
• Common parameters

Versioning

This library uses a simplified semantic versioning scheme: major version change for changes that are not backwards compatible (not expecting these) and the minor version for all backwards compatible changes. We won't make the distinction between new functionality and bugfixes, since we don't think it matters and is not a thing worth wasting time on. We will release often and if we need changes that are not comptatible, we will fork the next major version and release alphas versions of that until we are happy to release the next major version and try and have a painless upgrade path.

Dependencies

Python dependencies are specified in pyproject.toml. pip-compile is used to generate the requirements.txt file. To update the requirements, use the following commands:

pipx install pip-tools
pip-compile pyproject.toml

Dev and test requirements are specified separately:

# This will output the requirements for the "dev" and "testing" groups
pip-compile --all-extras --output-file=dev-requirements.txt --strip-extras pyproject.toml

Use the helper script to automatically run pip-compile:

./compile-requirements.sh

Running tests

Install test requirements:

pip install -r dev-requirements.txt

Run tests with Pytest:

python -m pytest -v .

Code formatting and linting

This project uses ruff together with isort for Python code formatting and linting. They are configured with a custom line length limit of 120.

Usage:

pipx install isort ruff
isort .
ruff check --fix .
ruff format .

Using with pre-commit:

# setup to be run automatically on git commit
pre-commit install

# run manually
pre-commit run --all-files