prudentia 2.6

Continuous Deployment toolkit.

Prudentia is a Continuous Deployment toolkit written in Python.

Mission

Prudentia’s mission is to help you to get production (or any other environment) ready in minutes instead of days, by streamlining all the actions needed to provision your architectural components.

Features

Prudentia uses Ansible as its main automation system, so it easily understands Ansible playbooks. A playbook is one of the components needed to define a Prudentia Box.

Prudentia currently offers:

• a CLI (supporting auto-completion) used to interactively define Boxes and run operations on them

• Here-Document format to script Prudentia environments

• provisioning of an existing server that can be accessed trough SSH

• management of the lifecycle of a Box that has been created through Prudentia

• creating Boxes using one of these providers:

  • Vagrant

  • DigitalOcean

  • local

  • ssh

Currently, all features work with Python 2.7+ and 3.4+.

Prerequisites

You need at minimum:

• Python 2.7 and pip

To install on a Linux distribution you need:

• libffi-dev

• libssl-dev

• python-dev

Installation

To install Prudentia:

$ pip install prudentia

It may be necessary to have root privileges, in which case:

$ sudo pip install prudentia

To uninstall:

$ pip uninstall prudentia

Box operations

A Simple provider (e.g. Local provider or SSH provider) have the following operations available:

• register: adds a new box definition to the registry

• unregister: removes a box from the registry

• reconfigure: changes the definition of an existing box

• list: lists all boxes in the registry

• set: defines or override an Ansible extra variable

• unset: removes an Ansible extra variable

• vars: loads Ansible extra variables from an external .yml or .json file (overriding existing ones)

• envset: sets the value of an environment variable

• provision: runs tasks defined in the playbook associated with a box

• decrypt: sets the password used to decrypt Ansible vault files

• verbose: sets Ansible verbosity, using a value between 0 and 4

• facts: shows useful information about the box and accepts optional parameter to filter properties

A Factory provider (e.g. Vagrant provider or DigitalOcean provider) extend simple provider and adds the ability to change the box life cycle:

• create: instantiate a new instance based of the box definition

• restart: reloads the instance

• stop: shuts down the instance

• destroy: kill the instance

• phoenix: shortcut for stop -> destroy -> create -> start -> provision (citing phoenix server Martin Fowler’s article)

• status: returns the status of the instance

Usage

CLI

We’ll show a usage example of the SSH provider bundled with Prudentia.

Make sure you have a server that you can ssh into.

$ prudentia ssh

Check what the Ssh provider can do using tab completion:

(Prudentia > Ssh)
decrypt      EOF          help         list         provision    reconfigure  register     set          unregister   unset        vars

Let’s start registering a new box:

(Prudentia > Ssh) register
Specify the playbook path:

Now Prudentia is asking for a playbook path, and this is actually an Ansible playbook.

You can use one of the samples that you can find in the examples/boxes directory. For instance, the tasks.yml that will run some Ansible tasks that we’ve defined (those tasks are not that meaningful, but they are used as a sanity check in our tests).

So let’s continue using the tasks.yml:

(Prudentia > Ssh) register
Specify the playbook path: /path/to/prudentia/examples/boxes/tasks.yml
Specify the box name [default: tasks-host]:
Specify the instance address or inventory: ip.of.your.server
Specify the remote user [default: _your_user_]:
Specify the password for the remote user [default: ssh key]:

Box example -> (/path/to/prudentia/examples/boxes/tasks.yml, tasks-host, ip.of.your.server, _your_user_) added.

You will notice that, for some questions, Prudentia gives suggested answer within [ ]. For instance, the suggested Box name is tasks-host. If you like the suggestion, just press enter to choose it.

So far we’ve registered a Prudentia Box that can be used to play around. If you want to check the definition again:

(Prudentia > Ssh) list
example -> (/path/to/prudentia/examples/boxes/tasks.yml, tasks-host, ip.of.your.server, _your_user_)

Now that we have double-checked that our Box has been registered, we can provision it:

(Prudentia > Ssh) provision example

PLAY [tasks-host] ***************************************************************

GATHERING FACTS ***************************************************************
ok: [tasks-host]

TASK: [Uname] *****************************************************************
changed: [tasks-host] => {"changed": true, "cmd": ["uname", "-a"], "delta": "0:00:00.005527", "end": "2015-01-01 19:13:58.633534", "rc": 0, "start": "2015-01-01 19:13:58.628007", "stderr": "", "stdout": "Darwin tiziano-air 12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64", "warnings": []}

TASK: [Shuffle] ***************************************************************
ok: [tasks-host] => (item=2) => {
    "item": 2,
    "msg": "2"
}
ok: [tasks-host] => (item=4) => {
    "item": 4,
    "msg": "4"
}
ok: [tasks-host] => (item=1) => {
    "item": 1,
    "msg": "1"
}
ok: [tasks-host] => (item=5) => {
    "item": 5,
    "msg": "5"
}
ok: [tasks-host] => (item=3) => {
    "item": 3,
    "msg": "3"
}

TASK: [No operation] **********************************************************
ok: [tasks-host] => {
    "msg": "Task noop executed."
}

PLAY RECAP ********************************************************************
tasks-host                  : ok=4    changed=1    unreachable=0    failed=0

Play run took 0 minutes

Now Prudentia has done the reasonable uninteresting uname, shuffling a list of ints and noop tasks on the remote machine.

Here-Document

The same sequence of operations can be executed using the Here-Document input:

$ prudentia ssh <<EOF
register
/path/to/prudentia/examples/boxes/tasks.yml
tasks-host
ip.of.your.server
_your_user_

provision tasks-host

unregister tasks-host
EOF

Command arguments

If you want to run few commands that don’t require specific inputs then there is an option that is quicker than using the CLI or the Here-Document.

Let’s for example have a look at an example right away:

$ prudentia ssh 'decrypt' 'vars ./encrypted-vars.yml' 'provision box-name'

After running this command we will be asked to input the Ansible vault password, after that an encrypted file containing variables will be loaded (we assume that the provided password can correctly decrypt the file) and eventually provision an existing registered ssh box.

Development

You can debug and extend Prudentia (or run the latest develop) simply by sym-linking a bash script that we provided:

$ ln -s `pwd`/prudentia.sh /usr/local/bin/prudentia-dev
$ prudentia-dev

In this way you can have both versions, stable and development, running on your system. The development version will run in a python virtual environment without interfering with the dependencies of the stable version. The only information that will be shared are the boxes definition.

More

Posts

Here you can find a guide on how to use Prudentia to provision a Digital Ocean droplet with the StarterSquad website on it.

Another important source of information is Iwein’s post that gives you an idea of what Continuous Delivery is, and where Prudentia fits into the flow.

Questions & Contributions

Questions, Contributions and Feedback are more than welcome.

You can checkout planned new features on the Trello Board. Feel free to create feature requests on github issues.

You can e-mail me at:

[e-mail]

Release History

2.6 (2019-02-10)

Improvements

• Support for picking Ansible vault-ids from default locations.

Security

• Upgrade pyOpenSSL.

2.5 (2017-11-28)

Improvements

• Enable support for Ansible 2.4.x.

• Update Java JRE version to 8u151.

2.4 (2017-08-31)

Improvements

• Adds bundled task: MySQL 5.7.

• Adds bundled task: raw Python2.

• Use JRE 8u131, use ansible modules where it makes sense, more readable code style.

Bugfixes

• Fix issue with passing only tags to Ansible.

2.3 (2016-12-20)

Improvements

• Adds bundled task: Node Yarn.

Bugfixes

• Fixes Digital Ocean image listing.

• Makes HashiVault lookup and action plugins compatible with Ansible 2.

• Timezone: Fix hour definition.

• GitHub: Workaroud for ssh_dir recursive error.

2.2 (2016-06-28)

Improvements

• Upgrades to support Ansible 2.1.

Bugfixes

• Mongodb_3: fetch correct apt key.

• Elasticsearch: makes daemon automatically startup after server reboot.

2.1 (2016-05-16)

Improvements

• Replace sudo with become in bundled tasks.

• Makes phoenix operation accept tags.

• Parametrise add-sudo-user bundled task to use ssh key of the specified user.

• Addresses deprecation warning for JRE bundled task.

Bugfixes

• Makes sure verbose operation correctly works.

2.0 (2016-04-03)

Improvements

• Upgrades to support Ansible 2.

• Sets user real name when creating sudo user using bundled task.

• Adds optional parameter root_mail_address to postfix bundled task.

Bugfixes

• Fixes timezone bundled task to avoid ntpdate running every minute.

1.0 (2016-02-09)

Improvements

• Allows specifying version for mongodb_3 bundled task.

• Avoids dependency from Ansible constants module.

• Changes default logging level.

Bugfixes

• Returns valid cli completions when multiple box names with same prefix are available.

0.17.1 (2016-01-04)

Improvements

• Introduces parametrize ntp server address for timezone bundled task.

• Removes initial warning message when creating environment.

• Disables output variables sets according to verbosity.

• Adds six as dependency.

• Provides backwards compatibility to java7 bundled task.

Bugfixes

• Makes verbose command resilient.

• Catch errors when parsing playbook on env loading.

0.17 (2015-12-04)

Improvements

• Adds facts CLI action that can be used to show information gathered from a box.

• Allows jre bundled task to provision a different java version.

• Digital-Ocean provider: prints image distribution as well when listing images.

• Digital-Ocean provider: uses image slug for default image instead of id.

Bugfixes

• Avoids use of getpass when inputing sensible information through heredoc.

• Digital-Ocean provider: not suggests default ubuntu image if not available within the images list.

0.16.1 (2015-11-19)

Bugfixes

• Update apt cache after adding ubuntu repositories.

• Installs correctly prudentia when using the homonym task.

0.16 (2015-11-19)

Improvements

• Removes update-cache from all apt tasks.

• Updates to SBT 0.13.9, nvm 0.29, node 0.12.

• Revisions task and file namings.

• Enhances project readme.

• Adds bundled tasks: jre, postfix.

Bugfixes

• Leverages Ansible play to get proper information that will be used by the box.

• Makes sure webdriver path is found, is dependant from node and adds start at the end of the installation.

0.15.1 (2015-10-02)

Bugfixes

• Digital Ocean: better error handling in case the target instance cannot be contacted.

• Digital Ocean: avoids misleading keys definition when registering an existing box.

• Uses correctly hostname as pattern during provisioning to instruct Ansible which instance to target.

• Adds hvac missing dependency used by Vault module and plugin.

Improvements

• Updates dependencies to latest version for development.

0.15 (2015-09-29)

Improvements

• Adds script that can generate dynamically an Ansible inventory based on the instances connected to an AWS ELB.

• Adds HashiCorp Vault Ansible lookup plugin.

• Adds HashiCorp File Ansible module.

• Adds bundled task: mongodb_3.

• Updates Ngnix example and improves Monit task.

• Updates to Ansible 1.9.3.

Bugfixes

• Changed state for UFW from ‘disabled’ to ‘reset’ to avoid old and new rules to be merged.

0.14 (2015-09-04)

Improvements

• Accepts now external inventory file, directory and script as alternative for the box address.

• Adds envset CLI action that can be used to define system environment variables.

• Disables Ansible verbose output and introduces verbose CLI action to explicit increase verbosity.

• Loads automatically vars/global.yml avoiding the need from now on to specify it in every playbook.

• Adds bundled tasks: sysdig, haproxy.

Bugfixes

• Fixes Digital Ocean droplet creation.

0.13 (2015-08-18)

Improvements

• Enable support for multiple base images on the Vagrant provider.

• List available base images when registering Vagrant box.

• Adds bundled tasks: vsftpd, mailhog, monit.

• Upgrades vault bundled task to 0.2

• Introduces retries mechanism when asking the user to provide a valid path.

Bugfixes

• Makes sure that Jinja2 templates do not ignore undefined variables and raise an error instead.

0.12 (2015-07-24)

Improvements

• Makes Nginx bundled task disable the default site.

• Shows more information about the DigitalOcean image when registering/reconfiguring a droplet.

• Allows only the newly added sudo user to not be prompted for password.

• Upgrades to Ansible 1.9.2.

• Adds bundled tasks: vault (https://vaultproject.io), fail2ban, tomcat7.

• Adds an action for the simple provider to set the password used to decrypt Ansible vault files.

• Refactors main cli to properly parse input arguments.

• Accepts list of commands as arguments.

• Introduces -v (–version) argument to print current Prudentia version.

• Adds an action for the simple provider to loads extra vars from an external .yml or .json file.

• Checks if current version is the latest released one.

• Accepts input paths relative to the directory where Prudentia was started or relative to the user home directory.

• Upgrades dopy to 0.3.6 and switches to DigitalOcean API version 2 based on API token.

Bugfixes

• Makes Nginx bundled task properly idempotent and reload the service at the end of the task.

• Fixes ElasticSearch init script.

• Makes sure variables value are set even if they contain spaces.

• Waits for async bash thread to finish.

• Fixes InsecurePlatformWarning when https connections are initiated.

Misc

• Updates Client component example.

• Moves build to new Travis container based infrastructure.

• Enables properly coverage verification and improved the coverage itself.

• Verifies support for Python 3.2+.

Documentation

• Adds decrypt action doc.

• Adds vars action doc.

• Extends Usage section describing the new Commands list argument.

0.11 (2015-06-19)

Improvements

• Suggests automatically latest Ubuntu 14.04 LTS 64bit image when creating DigitalOcean droplet.

• Validates setting extra variables and show existing ones when running unset without arguments.

• Updates examples.

• Adds bundled tasks: osquery, ufw, add sudo user, zeromq, elastic search, collectd, mongodb 2.6.

• Generalize bundled java task.

• Upgrades to a newer version of nginx using proper apt repository.

• Upgrades to SBT 0.13.8.

Bugfixes

• Sets correctly the user that will run the webdriver manager.

0.10 (2015-05-12)

Improvements

• Updates examples.

• Upgrade to Ansible 1.9.1.

Bugfixes

• Fixes buffering issue.

0.9.1 (2015-03-18)

Bugfixes

• Fixes issue if cli history file doesn’t exist.

0.9 (2015-03-18)

Improvements

• Enables cli history cross sessions.

• Adds bundled task for adding ssh known host.

• Adds status action for factory providers.

• Upgrade to Ansible 1.8.4.

• Improves examples.

• Increases code quality.

0.8.1 (2015-02-15)

Bugfixes

• Fixes tor bundled task.

Improvements

• Makes postgresql and sbt parametrized tasks.

• Improves ssh key bundled task using file module.

0.8 (2015-02-05)

Bugfixes

• Fixes shared folder definition for Vagrant box.

• Includes HISTORY in python setup manifest.

0.7 (2015-02-04)

Bugfixes

• Fixes stop recreation DigitalOcean droplet when user reconfigures box without destroying it.

Improvements

• Makes provision accept multiple tags.

• Suggests tags during auto-completion filtering out the ones that have already been selected.

• Enables symlinks feature in VirtualBox.

• Registers an existing DigitalOcean droplet using the id.

Misc

• Adds History and Authors.

0.6 (2015-01-07)

Bugfixes

• Fix creation user dir.

Documentation

• Described properly box operations.

0.5 (2015-01-07)

Bugfixes

• Fixes error when running an action against a non existing box.

Improvements

• Drops execution of the script to install Vagrant.

• Publishes Prudentia on PyPI.

• Adds Python 2.6 to Travis build options.

• Refactor nodejs bundled task to use nvm (#11).

• Hides password when user enters it during box definition (#10).

• Executes extra checks when user inputs file paths (#8).

• Updates Readme doc.

• Updates and cleans up examples.

• Creates Local Provider.

• Adds bundled tasks: fontforge, opencv, noop, postgres, sbt, ssl-self-certificate, timezone.

Behavioral Changes

• Restructures python packages.

• Moves Prudentia environments directory under user home.

• Avoids check and install Vagrant package when using Vagrant Provider.

Misc

• Adds license.

0.4 (2014-02-09)

Bugfixes

• Fixes several issue with Vagrantfile.

• Fixes provisioning non existing box.

Improvements

• Adds set/unset action used to set an environment variable.

• Sets default for yes/no question if no answer was given.

• Integrates Travis CI.

• Suggest box name based on playbook hosts name.

• Exit with error code 1 if one off cmd provisioning fails.

• Add example box.

0.3 (2014-01-16)

Improvements

• Creates DigitalOcean Provider and Ssh Provider.

• Introduces Environment and Box entities.

• Adds bundled tasks: chrome, protractor, mongodb, python.

• Introduces bash utility.

0.2 (2013-10-15)

Bugfixes

• Fixes provision without tags.

Improvements

• Loads box playbook tags and use in action argument suggestion.

0.1 (2013-09-17)

Beginning

• Adds script to install Vagrant and Ansible.

• Creates Vagrant Provider with basic commands: add, remove, provision, phoenix, restart, destroy.

• Adds bundled tasks: common-setup, git, github, java7, jenkins, mercurial, mysql, nginx, nodejs, redis, ruby, sbt, ssh-key, tor.

• Provides tags support for provision action.

• Adds shared folder to Vagrant box definition.