Python library to expose S3 as vault to store encrypted data
Project description
S3Vault Library
Python library to expose S3 as vault to store encrypted data
Goal
When you need to deploy some server you always want to have a vault where to store the keymaterials used by the server.
Usually this means you need to deploy a sort of vault server to manage access control and encryption (like Vault from Hashicorp). But also this service need some secret to be stored.
With AWS you can leverage on KMS + Policies and Roles to setup the access control and the permission to use encryption key. Then part of the job is already done and can be automatised with cloudformation.
For the remaining part (provisioning the keymaterials and the retrieval from within the instance) you can use S3Vaultlib
Features
The library let you store in an encrypted way, keymaterials into S3, by using server side encryption and different keys per role.
The main feature of the library though, is the capability to abstract S3 in a way you can use it as key/value store or blob store in jinja templates while you are provisioning your instance.
S3Vaultlib comes also with a simple cli, s3vaultcli that help to automate some simple tasks.
Ansible Module
The library includes also a useful ansible module that allows you to easily create files from templates. Check the following example:
template j2:
template test {{ nginx.server_name }} port: {{ nginx.port }} certificate: {{ cert }} htpasswd: {{ htpasswd }} environment: {{ ansible_env.PYENV_SHELL }} environment2: {{ environment['LOGNAME'] }}
playbook:
--- - name: test my new module connection: local hosts: localhost tasks: - name: test s3vault_template: bucket: 230706054651 path: vault/nginx/ kms_alias: gchiesa/testkey src: template.j2 dest: outcome.txt ec2: false region: eu-west-1
This way the s3vault_template module will take the template, connect to S3, expose the filesystem as variables and you use the files in your template
Example Use Case
We need to deploy a nginx instance, we need to provision a server name and a port dynamically and the htpasswd file for basic authentication.
First of all we will have a KMS key with the name (nginx-key) for the role associated to the instance (nginx-role)
Now we provision the secrets in the vault with the cli:
nginx configuration:
s3vaultcli -b <bucket> -p vault/nginx configset -c nginx -K server_name -V www.example.com s3vaultcli -b <bucket> -p vault/nginx configset -c nginx -K server_port -V 8443
htpasswd upload:
s3vaultcli -b <bucket> -p vault/nginx push -s htpasswd -d htpasswd
NOTE: the library will try to detect the role and use a KMS key with the same alias of the role name. If we are in another machine (or from our local machine we need to have access to the KMS key and specify the alias with the -k key_alias option)
In S3 now, we will have a structure like this:
$ aws s3 ls s3://<bucket>/vault/nginx/ 2017-08-20 18:02:10 5 htpasswd 2017-08-20 18:00:39 57 nginx
Now inside the instance we can prepare the templates to expand, for nginx.conf.j2
$ cat nginx.conf.j2 server { listen {{ nginx.server_port }}; server_name {{ nginx.server_name }}; access_log logs/localhost.access.log main; location / { root html; index index.html index.htm; } include /etc/nginx/sites-enabled/*; } }
And the htpasswd.j2:
$ cat htpasswd.j2 {{ htpasswd }}
When the instance starts in the userdata you can use the s3vaultcli tool to render the templates, in this way:
s3vaultcli -b <bucket> -p vault/nginx template -t nginx.conf.j2 -d nginx.conf s3vaultcli -b <bucket> -p vault/nginx template -t htpasswd -d htpasswd
License
Free software: BSD license
Documentation: https://s3vaultlib.readthedocs.io.
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.0 (2017-08-19)
First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.