tools to support builds combining gopass, terraform, dda-pallet, aws & hetzner-cloud
Project description
dda-devops-build
| team@social.meissa-gmbh.de | Website & Blog
dda-devops-build provide a environment to tie several DevOps tools together for easy interoperation. Supported tools are:
- aws with
- simple api-key auth
- mfa & assume-role auth
- hetzner with simple api-key auth
- terraform v0.11, v0.12 supporting
- local file backends
- s3 backends
- docker / dockerhub
- user / team credentials managed by gopass
- dda-pallet
Setup
Ensure that yout python3 version is at least Python 3.7!
sudo apt install python3-pip
pip3 install pip3 --upgrade
pip3 install pybuilder ddadevops deprecation
export PATH=$PATH:~/.local/bin
# in case of using terraform
pip3 install dda-python-terraform packaging
# in case of using AwsMixin
pip3 install boto3
# in case of using AwsMfaMixin
pip3 install boto3 mfa
Example Build
lets assume the following project structure
my-project
| -> my-module
| | -> build.py
| | -> some-terraform.tf
| -> an-other-module
| -> target (here will the build happen)
| | -> ...
from pybuilder.core import task, init
from ddadevops import *
name = 'my-project'
MODULE = 'my-module'
PROJECT_ROOT_PATH = '..'
class MyBuild(DevopsTerraformBuild):
pass
@init
def initialize(project):
project.build_depends_on('ddadevops>=0.5.0')
account_name = 'my-aws-account-name'
account_id = 'my-aws-account-id'
stage = 'my stage i.e. dev|test|prod'
additional_vars = {'var_to_use_insied_terraform': '...'}
additional_var_files = ['variable-' + account_name + '-' + stage + '.tfvars']
config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
MODULE, additional_vars,
additional_tfvar_files=additional_var_files)
build = MyBuild(project, config)
build.initialize_build_dir()
@task
def plan(project):
build = get_devops_build(project)
build.plan()
@task
def apply(project):
build = get_devops_build(project)
build.apply()
@task
def destroy(project):
build = get_devops_build(project)
build.destroy()
@task
def tf_import(project):
build = get_devops_build(project)
build.tf_import('aws_resource.choosen_name', 'the_aws_id')
Feature aws-backend
Will use a file backend.dev.live.properties
where dev is the [account-name], live is the [stage].
the backend.dev.live.properties file content:
key = ".."
region = "the aws region"
profile = "the profile used for aws"
bucket = "the s3 bucket name"
kms_key_id = "the aws key id"
the build.py file content:
class MyBuild(AwsBackendPropertiesMixin, DevopsTerraformBuild):
pass
@init
def initialize(project):
project.build_depends_on('ddadevops>=1.0')
account_name = 'my-aws-account-name'
account_id = 'my-aws-account-id'
stage = 'my stage i.e. dev|test|prod'
additional_vars = {}
config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
MODULE, additional_vars)
config = add_aws_backend_properties_mixin_config(config, account_name)
build = MyBuild(project, config)
build.initialize_build_dir()
Feature aws-mfa-assume-role
In order to use aws assume role in combination with the mfa-tool (pip install mfa
):
the build.py file content:
class MyBuild(class MyBuild(AwsMfaMixin, DevopsTerraformBuild):
pass
@init
def initialize(project):
project.build_depends_on('ddadevops>=1.0')
account_name = 'my-aws-account-name'
account_id = 'my-aws-account-id'
stage = 'my stage i.e. dev|test|prod'
additional_vars = {}
config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
MODULE, additional_vars)
config = add_aws_backend_properties_mixin_config(config, account_name)
config = add_aws_mfa_mixin_config(config, account_id, 'eu-central-1',
mfa_role='my_developer_role',
mfa_account_prefix='company-',
mfa_login_account_suffix='users_are_defined_here')
build = MyBuild(project, config)
build.initialize_build_dir()
@task
def access(project):
build = get_devops_build(project)
build.get_mfa_session()
Feature DdaImageBuild
The docker build supports image building, tagging, testing and login to dockerhost. For bash based builds we support often used script-parts as predefined functions see install_functions.sh.
A full working example: doc/example/50_docker_module
Feature AwsRdsPgMixin
The AwsRdsPgMixin provides
- execute_pg_rds_sql - function will optionally resolve dns-c-names for trusted ssl-handshakes
- alter_db_user_password
- add_new_user
- deactivate_user
the build.py file content:
class MyBuild(..., AwsRdsPgMixin):
pass
@init
def initialize(project):
project.build_depends_on('ddadevops>=1.0')
...
config = add_aws_rds_pg_mixin_config(config,
stage + "-db.bcsimport.kauf." + account_name + ".breuni.de",
"kauf_bcsimport",
rds_resolve_dns=True,)
build = MyBuild(project, config)
build.initialize_build_dir()
@task
def rotate_credentials_in(project):
build = get_devops_build(project)
build.alter_db_user_password('/postgres/support')
build.alter_db_user_password('/postgres/superuser')
build.add_new_user('/postgres/superuser', '/postgres/app', 'pg_group_role')
@task
def rotate_credentials_out(project):
build = get_devops_build(project)
build.deactivate_user('/postgres/superuser', 'old_user_name')
Releasing and updating
Publish snapshot
- every push will be published as dev-dependency
Release
adjust version no in build.py to release version no.
git commit -am "release"
git tag -am "release" [release version no]
git push --follow-tags
increase version no in build.py
git commit -am "version bump"
git push
pip3 install --upgrade --user ddadevops
License
Copyright © 2021 meissa GmbH Licensed under the Apache License, Version 2.0 (the "License")
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file ddadevops-4.0.0.dev43.tar.gz
.
File metadata
- Download URL: ddadevops-4.0.0.dev43.tar.gz
- Upload date:
- Size: 30.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a70a464b596bb3ce08cfffae3d37877a896c0746e4cfc0d1f524b20e72f5181b |
|
MD5 | bfa05c09985e110fadcefa95896b2509 |
|
BLAKE2b-256 | 46ce73563b6a37b9430ca21cfe67b846c8e5ec4ca66542d791814eedf58023d0 |
File details
Details for the file ddadevops-4.0.0.dev43-py3-none-any.whl
.
File metadata
- Download URL: ddadevops-4.0.0.dev43-py3-none-any.whl
- Upload date:
- Size: 40.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a09c5b462c5f423b3b72aab86ff0932e36c54bcdb86cd0e263b1e23a3354bf5e |
|
MD5 | 9afa40883747d7d20cebeda52bcaa8f0 |
|
BLAKE2b-256 | e9258e52b1ee497af34041afd3332d14ededb3bab42434c46bf37687feb9fd30 |