This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Ministry of Justice - Bootstrap Salt

The aim of this repos is to provide a standard way to bootstrap salt master and minions in an AWS environment. Currently it depends on bootstrap-cfn

It exposes a fabric task called salt.setup which will elect a master from an Auto Scaling Group (ASG) and install minions on any other members of the ASG.

Installation

git clone git@github.com:ministryofjustice/bootstrap-salt.git
cd bootstrap-salt
pip install -r requirements.txt

Developing and running tests

The test suite can be run via setup.py as follows:

python -m unittest discover

or:

python setup.py test

Example Usage

Bootstrap-salt uses fabric

If you also want to bootstrap the salt master and minions, you can do this:

fab application:courtfinder aws:prod environment:dev config:/path/to/courtfinder-dev.yaml salt.setup
  • application:courtfinder - should match the name given to bootstrap-cfn
  • aws:dev - is a way to differentiate between AWS accounts (~/.config.yaml)
  • environment:dev - should match the environment given to bootstrap-cfn
  • config:/path/to/file.yaml - The location to the project YAML file

Multiple Stacks

If you want to run multiple stacks with the same name, ensure that you specify a zone for the salt master DNS records in the yaml config:

master_zone:
  my-zone.dsd.io

Then when you create a stack you can specify a tag before cfn_create, like:

fab application:courtfinder aws:my_project_prod environment:dev config:/path/to/courtfinder-dev.yaml tag:active cfn_create salt.setup

NB active is the default.

Then you can refer to this stack by it’s tag in the future. In this way it is easier to bring up two stacks from the same config. If you want to swap the names of the stacks you can do the following:

fab application:courtfinder aws:my_project_prod environment:dev config:/path/to/courtfinder-dev.yaml swap_tags:inactive,active salt.swap_masters:inactive,active

Example Configuration

AWS Account Configuration

This tool needs AWS credentials to create stacks and the credentials should be placed in the ~/.aws/credentials file (which is the same one used by the AWS CLI tools). You should create named profiles like this (and the section names should match up with what you specify to the fabric command with the aws:my_project_prod flag)

[my_project_dev]
aws_access_key_id = AKIAI***********
aws_secret_access_key = *******************************************
[my_project_prod]
aws_access_key_id = AKIAI***********
aws_secret_access_key = *******************************************

If you wish to authenticate to a separate AWS account using cross account IAM roles you should create a profile called cross-account with the access keys of the user with permission to assume roles from the second account:

[cross-account]
aws_access_key_id = AKIAI***********
aws_secret_access_key = *******************************************

And when you run the tool you must set the ARN ID of the role in the separate account which you wish to assume. For example:

AWS_ROLE_ARN_ID='arn:aws:iam::123456789012:role/S3Access' fab application:courtfinder aws:prod environment:dev config:/path/to/courtfinder-dev.yaml salt.setup

Salt master DNS

This tool will attempt to create a DNS entry for the salt master during the bootstrap process. You must specify the zone in which to create the entry in the bootstrap-cfn config file like so:

master_zone: myzone.dsd.io

If you do not specify a zone, then no entry will be created and you will need AWS credentials as above to discover the master when deploying etc.

The entry created will be:

master.<environment>.<application>.<master_zone>

Salt specific configuration

In order to rsync your salt states to the salt master you need to add a salt section to the top level of your project’s YAML file. The following parameters specify the rsync sources and targets:

  • local_salt_dir: Directory containing all the files you want to have in your salt root (for example top.sls or project specific states).

    Default value: ./salt

  • local_pillar_dir: Directory containing all the files you want to have in your pillar root.

    Default value: ./pillar

  • local_vendor_dir: Directory containing formulas cloned by salt-shaker.

    Default value: ./vendor

  • remote_state_dir: Salt root on the master.

    Default value: /srv/salt

  • remote_pillar_dir: Pillar root on the master.

    Default value: /srv/pillar

The cloudformation yaml will be automatically uploaded to your pillar as cloudformation.sls. So if you include -cloudformation in your pillar top file you can do things like:

salt-call pillar.get s3:static-bucket-name

Github based SSH key generation

To add individual users to the AWS stack.

  1. to customize the list of users, teams and keys, add the following to the project YAML template; it offers more flexibility: for example multiple keys per user, or limiting to specific keys for users with multiple keys:
myenv:
  github_users:
    ministryofjustice: # or any org
      individuals:
        - koikonom:
            fingerprints:
              - '35:53:6f:27:fe:39:8b:d8:dd:87:19:f3:40:d2:84:6a'
            unix_username:
              kyriakos
        - ashb:
            fingerprints:
              - '0c:11:2b:78:ff:8d:5f:f0:dc:27:8e:e2:f8:2f:ab:25'
              - 'af:e0:6c:dc:bd:9b:bf:1d:9b:de:2d:de:12:6e:f2:8a'
        - mattmb
      teams:
        - some-team-name
          - some-user:
              unix_username: userunixusername
              fingerprints: 00:11:22:33:44:55:66
        - anotherteam
  1. in the simplest version, just add the list of teams (all users’ keys will be used) to the projec YAML template
github_users:
  ministryofjustice: # or any org
    teams:
      - webops
      - crime-billing-online
  1. if a team doesn’t exist, create it on GitHub granting “Read” access. Example of a team:

    https://github.com/orgs/ministryofjustice/teams/webops

  2. obtain a GitHub token and set the GH_TOKEN variable in your environment:

    https://help.github.com/articles/creating-an-access-token-for-command-line-use/

  3. run the following command:

fab application:<yourapp> aws:<your_aws_profile> environment:myenv config:<your template yaml file> ssh_keys
  1. the above command, if succesful, creates the file pillar/<myenv>/keys.sls (can be renamed, often as admins.sls)

    Add an entry with the name of this file to pillar/<myenv>/top.sls

base:
  'Env:demo':
    - match: grain
    - demo
    - demo-secrets
    - cloudformation
    - admins
  1. highstate the stack
Release History

Release History

1.3.3

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.3.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.3.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
bootstrap_salt-1.3.3.tar.gz (24.3 kB) Copy SHA256 Checksum SHA256 Source Oct 4, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting