Skip to main content

State mangement tool for Future Agile CICD

Project description

AgileUp State

Python 3.8+ project to manage AgileUP pipeline states with the following features:

  • Linux and Windows compatible project.
  • Defines state model.
  • Saves and fetches states from vault.
  • Exports private key for Linux SSH connections.
  • Exports client PKI data for Windows WinRM connections.
  • Creates cloud init zip file for mTLS connection data to Windows WinRM hosts.
  • Exports ansible inventories for both Linux(SSH) and Windows(WinRM) connections.
  • Provides simple connectivity tests.


This project uses poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on, it will manage (install/update) them for you.

Use the installer rather than pip installing-with-the-official-installer.

poetry self add poetry-bumpversion
poetry -V
Poetry (version 1.2.0)

Windows Path

Install poetry from powershell in admin mode.

(Invoke-WebRequest -Uri -UseBasicParsing).Content | py -

The path will be C:\Users\<YOURUSER>\AppData\Roaming\Python\Scripts\poetry.exe which you will need to add to your system path.

Windows GitBash

When using gitbash you can setup an alias for the poetry command:

alias poetry="\"C:\Users\<YOURUSER>\AppData\Roaming\Python\Scripts\poetry.exe\""

Getting Started

poetry update
poetry install


This project uses the hvac python module and to develop locally you can run vault as a docker service as detailed here local docker vault. For local development vault setup follow the VAULT guide for information.

Check your connection with the following command, note in development mode vault should not be sealed.

export VAULT_ADDR='http://localhost:8200'
export VAULT_TOKEN='8d02106e-b1cd-4fa5-911b-5b4e669ad07a'

poetry run agileupstate check

Required Environment State Variables

Variable Description
SIAB_ID Unique environment ID.
SIAB_CLOUD Cloud vendor API mnemonic.
SIAB_LOCATION1 Cloud vendor defined cloud location, uksouth, etc.
SIAB_LOCATION2 Cloud vendor defined cloud location, UK South, etc.
SIAB_CONTEXT Environment context, e.g. dev. test, prod.
SIAB_VALUES_PATH Vault path to common environment values to be exported.
SIAB_DOMAIN Optional public domain that overrides cloud provided DNS names.

SIAB_LOCATION: Azure has a different location string between "accounts" and "resources" and only uksouth is useful to the automation, but we must also provide UK South for resource groups. FIXME: Needs to be verified.

SIAB_VALUES_PATH: Rather than load variables into the delivery platform environment, there can be many, a better option is to define a YML file that contains all the required common variables for a specific environment and have the user upload that to vault. This application can then download the YML data file and convert it into an exports file that can be sourced by the pipelines. These environment values that are exported can then be used by this project and other utilities such as terraform, ansible and powershell.

SIAB_DOMAIN: Cloud DNS services might in some cases provide a DNS domain that is not the same as the public internet domain required by the project, for example might optionally be

username/password: These values are common across the environment, for example Ubuntu Azure images use a username=azureuser, and so it simplifies configuration if the same credentials are used for Linux and Windows environments running in Azure for administration access, client administration access as well as PFX certificate files used on Windows for WinRM certificate authentication. For AWS ubuntu is the username for Ubuntu images the same approach can be taken there.

Required environment inputs:

These values should be setup in your CD platforms environment variables.

export SIAB_ID=001
export SIAB_CLOUD=arm
export SIAB_LOCATION1=uksouth
export SIAB_LOCATION2="UK South"
export SIAB_CONTEXT=dev
export SIAB_VALUES_PATH=siab-state/001-arm-uksouth-dev/siab-values/siab-values.yml

Required values inputs (stored in vault path SIAB_VALUES_PATH):

  username: azureuser
  password: mypassword
  ca_trust_path: siab-client/
  cert_pem: siab-client/
  cert_key_pem: siab-client/
  group_owner: Paul Gilligan
  group_department: DevOps
  group_location: uksouth

Required Supporting Data

Some data is generated only once and thus can be uploaded to vault manually.

Uploading values file:

base64 ./siab-values.yml | vault kv put secret/siab-state/001-arm-uksouth-dev/siab-values/siab-values.yml file=-

Uploading pfx files:

base64 ./ | vault kv put secret/siab-pfx/ file=-
base64 ./ | vault kv put secret/siab-pfx/ file=-

Uploading pki files:

base64 ./ | vault kv put secret/siab-client/ file=-
base64 ./ | vault kv put secret/siab-client/ file=-
base64 ./ | vault kv put secret/siab-client/ file=-

Provision Use Case

Example steps required for the Windows terraform provision use case shown below.

agileupstate cloud-init --server-path=siab-pfx/ --client-path=siab-pfx/
agileupstate create
source ./                                                
terraform init
terraform apply -auto-approve
agileupstate save

Example steps required for the Linux terraform provision use case shown below.

agileupstate create
source ./                                                
terraform init
terraform apply -auto-approve
agileupstate save

Destroy Use Case

Example steps required for recovering system state use case shown below which might be for example to destroy an environment.

agileupstate load
source ./                                                
terraform init
terraform destroy -auto-approve

Ansible Use Case

Example steps required for the Windows ansible use case shown below.

agileupstate load
source ./                                                  
agileupstate inventory-windows --ca-trust-path=siab-client/ --cert-pem=siab-client/ --cert-key-pem=siab-client/
ansible-inventory -i inventory.ini --list
ansible -i inventory.ini "${TF_VAR_siab_name_underscore}" -m win_ping

Example steps required for the Linux ansible use case shown below.

agileupstate load
source ./                        
agileupstate inventory-linux
ansible-inventory -i inventory.ini --list
ANSIBLE_HOST_KEY_CHECKING=True ansible -i inventory.ini --user "${TF_VAR_admin_username}" "${TF_VAR_siab_name_underscore}" -m ping

Exports Use Case

The yml file from SIAB_VALUES_PATH is exported to the file with the contents as shown in the example below which can then be used by downstream utilities.

export SIAB_URL=
export SIAB_USERNAME=azureuser
export SIAB_PASSWORD=mypassword
export SIAB_CA_TRUST_PATH=siab-client/
export SIAB_CERT_PEM=siab-client/
export SIAB_CERT_KEY_PEM=siab-client/
export TF_VAR_group_owner=Paul Gilligan
export TF_VAR_group_department=DevOps
export TF_VAR_group_location=UK South
export TF_VAR_admin_username=azureuser
export TF_VAR_admin_password=mypassword
export TF_VAR_siab_name=001-arm-uksouth-dev
export TF_VAR_siab_name_underscore=001_arm_uksouth_dev
source ./

Cloud Init Data Use Case

Example cloud init command that generates the zip file that is loaded onto Windows machines for WimRM certificate authentication.

poetry run agileupstate cloud-init --server-path=siab-pfx/ --client-path=siab-pfx/

Ansible Windows Inventory Use Case

inventory.ini is generated with the target(s) and configuration information for a successful SSH connection from Ansible.

When export


Ansible Linux Inventory Use Case

inventory.ini is generated with the target(s) and configuration information for a successful SSH connection from Ansible.

When export

[001_arm_uksouth_dev] ansible_ssh_private_key_file=vm-rsa-private-key.pem


poetry run agileupstate


poetry run flake8


poetry run pytest


  • By default we are using PYPI packages.
  • Create yourself an access token for PYPI and then follow the instructions.
export PYPI_USERNAME=__token__ 
export PYPI_PASSWORD=<Your API Token>
poetry publish --build --username $PYPI_USERNAME --password $PYPI_PASSWORD


We use SemVer for versioning. For the versions available, see the tags on this repository.


We are using poetry-bumpversion to manage release versions.

poetry version patch


Once the release has been created it is now available for you to use in other python projects via:

pip install agileupstate

And also for poetry projects via:

poetry add agileupstate


Please read for details on our code of conduct, and the process for submitting pull requests to us.


This project is licensed under the Apache License, Version 2.0 - see the LICENSE file for details

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

agileupstate-0.0.44.tar.gz (14.7 kB view hashes)

Uploaded source

Built Distribution

agileupstate-0.0.44-py3-none-any.whl (12.9 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page