Skip to main content

tools to support builds combining gopass, terraform, dda-pallet, aws & hetzner-cloud

Project description

dda-devops-build

Slack | team@social.meissa-gmbh.de team@social.meissa-gmbh.de | Website & Blog

release prod

dda-devops-build provide a environment to tie several DevOps tools together for easy interoperation. Supported tools are:

  • aws with
    • simple api-key auth
    • mfa & assume-role auth
  • hetzner with simple api-key auth
  • terraform v0.11, v0.12 supporting
    • local file backends
    • s3 backends
  • docker / dockerhub
  • user / team credentials managed by gopass
  • dda-pallet

Setup

Ensure that yout python3 version is at least Python 3.7!

sudo apt install python3-pip
pip3 install pip3 --upgrade
pip3 install pybuilder ddadevops deprecation
export PATH=$PATH:~/.local/bin

# in case of using terraform
pip3 install dda-python-terraform packaging

# in case of using AwsMixin
pip3 install boto3

# in case of using AwsMfaMixin
pip3 install boto3 mfa

Example Build

lets assume the following project structure

my-project
   | -> my-module
   |       | -> build.py
   |       | -> some-terraform.tf
   | -> an-other-module
   | -> target  (here will the build happen)
   |       | -> ...
from pybuilder.core import task, init
from ddadevops import *

name = 'my-project'
MODULE = 'my-module'
PROJECT_ROOT_PATH = '..'

class MyBuild(DevopsTerraformBuild):
    pass


@init
def initialize(project):
    project.build_depends_on('ddadevops>=0.5.0')
    account_name = 'my-aws-account-name'
    account_id = 'my-aws-account-id'
    stage = 'my stage i.e. dev|test|prod'
    additional_vars = {'var_to_use_insied_terraform': '...'}
    additional_var_files = ['variable-' + account_name + '-' + stage + '.tfvars']
    config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
                                                  MODULE, additional_vars,
                                                  additional_tfvar_files=additional_var_files)
    build = MyBuild(project, config)
    build.initialize_build_dir()


@task
def plan(project):
    build = get_devops_build(project)
    build.plan()


@task
def apply(project):
    build = get_devops_build(project)
    build.apply()

@task
def destroy(project):
    build = get_devops_build(project)
    build.destroy()

@task
def tf_import(project):
    build = get_devops_build(project)
    build.tf_import('aws_resource.choosen_name', 'the_aws_id')

Feature aws-backend

Will use a file backend.dev.live.properties where dev is the [account-name], live is the [stage].

the backend.dev.live.properties file content:

key = ".."
region = "the aws region"
profile = "the profile used for aws"
bucket = "the s3 bucket name"
kms_key_id = "the aws key id"

the build.py file content:

class MyBuild(AwsBackendPropertiesMixin, DevopsTerraformBuild):
    pass


@init
def initialize(project):
    project.build_depends_on('ddadevops>=1.0')
    account_name = 'my-aws-account-name'
    account_id = 'my-aws-account-id'
    stage = 'my stage i.e. dev|test|prod'
    additional_vars = {}
    config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
                                                  MODULE, additional_vars)
    config = add_aws_backend_properties_mixin_config(config, account_name)
    build = MyBuild(project, config)
    build.initialize_build_dir()

Feature aws-mfa-assume-role

In order to use aws assume role in combination with the mfa-tool (pip install mfa):

the build.py file content:

class MyBuild(class MyBuild(AwsMfaMixin, DevopsTerraformBuild):
    pass


@init
def initialize(project):
    project.build_depends_on('ddadevops>=1.0')
    account_name = 'my-aws-account-name'
    account_id = 'my-aws-account-id'
    stage = 'my stage i.e. dev|test|prod'
    additional_vars = {}
    config = create_devops_terraform_build_config(stage, PROJECT_ROOT_PATH,
                                                  MODULE, additional_vars)
    config = add_aws_backend_properties_mixin_config(config, account_name)
    config = add_aws_mfa_mixin_config(config, account_id, 'eu-central-1',
                                      mfa_role='my_developer_role',
                                      mfa_account_prefix='company-',
                                      mfa_login_account_suffix='users_are_defined_here')
    build = MyBuild(project, config)
    build.initialize_build_dir()

@task
def access(project):
    build = get_devops_build(project)
    build.get_mfa_session()

Feature DdaDockerBuild

The docker build supports image building, tagging, testing and login to dockerhost. For bash based builds we support often used script-parts as predefined functions see install_functions.sh.

A full working example: doc/example/50_docker_module

Feature AwsRdsPgMixin

The AwsRdsPgMixin provides

  • execute_pg_rds_sql - function will optionally resolve dns-c-names for trusted ssl-handshakes
  • alter_db_user_password
  • add_new_user
  • deactivate_user

the build.py file content:

class MyBuild(..., AwsRdsPgMixin):
    pass


@init
def initialize(project):
    project.build_depends_on('ddadevops>=1.0')

    ...
    config = add_aws_rds_pg_mixin_config(config,
                                         stage + "-db.bcsimport.kauf." + account_name + ".breuni.de",
                                         "kauf_bcsimport",
                                         rds_resolve_dns=True,)
    build = MyBuild(project, config)
    build.initialize_build_dir()

@task
def rotate_credentials_in(project):
    build = get_devops_build(project)
    build.alter_db_user_password('/postgres/support')
    build.alter_db_user_password('/postgres/superuser')
    build.add_new_user('/postgres/superuser', '/postgres/app', 'pg_group_role')


@task
def rotate_credentials_out(project):
    build = get_devops_build(project)
    build.deactivate_user('/postgres/superuser', 'old_user_name')

Releasing and updating

Publish snapshot

  1. every push will be published as dev-dependency

Release

adjust version no in build.py to release version no.
git commit -am "release"
git tag -am "release" [release version no]
git push --follow-tags
increase version no in build.py
git commit -am "version bump"
git push
pip3 install --upgrade --user ddadevops

License

Copyright © 2021 meissa GmbH Licensed under the Apache License, Version 2.0 (the "License")

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ddadevops-4.0.0.dev8.tar.gz (27.1 kB view details)

Uploaded Source

Built Distribution

ddadevops-4.0.0.dev8-py3-none-any.whl (35.5 kB view details)

Uploaded Python 3

File details

Details for the file ddadevops-4.0.0.dev8.tar.gz.

File metadata

  • Download URL: ddadevops-4.0.0.dev8.tar.gz
  • Upload date:
  • Size: 27.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.7

File hashes

Hashes for ddadevops-4.0.0.dev8.tar.gz
Algorithm Hash digest
SHA256 175efecc0d10232b6c16dff10ecefe6892d77e5489cb465af2bfb91e45c35526
MD5 e8e59dae3dd4ea3c3de09cdffe1f14a3
BLAKE2b-256 53538d5fbb694a5a27c3bfd85f47540af2476613fd26647dc23fb244cfb45e41

See more details on using hashes here.

File details

Details for the file ddadevops-4.0.0.dev8-py3-none-any.whl.

File metadata

File hashes

Hashes for ddadevops-4.0.0.dev8-py3-none-any.whl
Algorithm Hash digest
SHA256 3bc12400c42cab3c9ecfc7fed4950a0447653ee266e660ecef1cd1ec24e0ac47
MD5 5a552b35cd8b400858a3ecfff288111b
BLAKE2b-256 ce432412354ae8eb2d172cca360f6e57cb088eaf877e2e25b70e6116b2ddec00

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page