vps-deploy 1.2.0

Common commands for deployment on a VPS.

This is a base set of Fabric 2 deployment functions for projects based on Django, Nginx, uWSGI Emperor, Memcached and PostgreSQL.

Fabric is a handy tool for automating deployment processes. Unlike Salt or Ansible, you’re writing code from the beginning rather than diving into a Turing Tarpit of templated YAML. Also unlike Salt, Ansible or shell scripts, Fabric neatly coordinates both local and remote SSH tasks. While Fabric may not be your best choice for provisioning, it works well as a simple and flexible tool for automatic custom workflows.

Installation (Fabric 2.*)

If Fabric 2 is available as an operating system package:

$ sudo apt install fabric invoke
$ pip install --user vps-deploy

Otherwise:

$ pip install --user fabric invoke vps-deploy

Getting started

To get started, create a fabfile.py in your top-level project directory. It might look something like this:

from fabric import task
from invoke.collection import Collection

from vps_deploy import django_fabric2 as df2

hosts = ['[e-mail]']

@task(hosts=hosts)
def deploy(c):
    df2.transfer_files_git(c)
    df2.init(c)
    df2.grep_for_pdb(c)
    df2.lint(c)
    df2.prepare_virtualenv(c)
    df2.prepare_django(c)
    df2.fix_permissions(c,
        # Sentry needs access to Git repo.
        read=[
            '.git', 'deploy', 'env', 'project'],
        read_write=['project', 'project/collected_static/CACHE'],
    )
    df2.reload_uwsgi(c)
    df2.flush_memcached(c)
    df2.update_nginx(c)

ns = Collection(
    deploy,
    task(df2.download_postgres_db, hosts=hosts),
    task(df2.mirror_postgres_db, hosts=hosts),
    task(df2.mirror_media, hosts=hosts),
)
ns.configure({
    # Built-in Fabric config.
    'run': {
        'echo': True,
        # Needed so local commands work. Can also use FABRIC_RUN_REPLACE_ENV.
        'replace_env': False,
        # Needed for Guix. Can also use FABRIC_RUN_SHELL.
        # 'shell': '/run/current-system/profile/bin/bash',
    },

    # Our custom project config.
    'env': {
        'branch': 'master',
        'app_user': 'www-data',
        'db_name': 'exampledb',
        'project_dir': '/srv/exampleproject',
        'media_dir': 'project/media',
        'virtualenv': '/srv/venvs/exampleproject-django-py38',
        'site_name': 'examplesite',
        'requirements': 'requirements/production.txt',
        'settings': 'project.settings.live',
        'uwsgi_conf': 'deploy/uwsgi.ini',
        'nginx_conf': 'deploy/nginx.conf',
        'python': '/usr/bin/python3.8',
    },
})

Deploying

To make a deployment:

fab –prompt-for-sudo-password deploy

This depends on a few things being already set up, such as SSH access to the server and having the server-site software and accounts set up. Those tend to to be better handled with configuration management tools like Salt or Ansible (and potentially triggered by Fabric!).

Release History

1.2.0 (2024-07-20)

• only search git-tracked files in grep_for_pdb command

• switch to f-strings throughout

• fix conditionals failing rather than returning False

• prevent Jinja2 from stripping trailing newlines (causes crontab issues)

• remove setup.py (not required with Setuptools v40.0.9 under normal use and Pip >= 21.1 when using --editable)

• add the ability to abort deployment when local git changes present

1.1.0 (2023-07-06)

• drop dependency on patchwork due to version conflict with Python 3.11-compatible version of invoke

• remove docs about Fabric 1

1.0.0 (2023-04-12)

• remove Python 2 support

• also look for breakpoint (Python 3.7) when checking for debugger calls

• make sudo_upload_template function part of public API

• use mktemp instead of deprecated tempfile

• add pip_install_options argument to prepare_virtualenv function

• checkout env.branch before running git reset

0.4.10 (2021-09-20)

Improvements

• switch to setup.cfg

0.4.9 (2021-09-20)

Improvements

• allow reading secrets from a GPG encrypted file with read_gpg_password_file

0.4.8 (2021-09-18)

Improvements

• switch to f-strings (pyupgrade –py36-plus)

• add homepage

0.4.7 (2021-06-29)

Improvements

• add transfer_files_git_pull command

0.4.6 (2021-06-22)

Improvements

• Allow grep_for_pdb to accept a comma-separated string of exclusions.

0.4.5 (2021-04-29)

Improvements

• prevent environment variables containing “%” due to uWSGI applying special meaning to this character

• add flake8_test and mypy_test commands

• switch to git ls-files '*.py' to avoid false-positives on non-versioned files

0.4.4 (2020-11-16)

Bug fix

• don’t attempt to restore database owner and privileges when running mirror_postgres_db

0.4.3 (2020-09-02)

Bug fix

• fix issue AttributeError: Raw error caused by dependencies

0.4.2 (2020-08-13)

Improvements

• use python3 -m to run pylint to avoid issues with Debian binary pylint3 and others using just pylint

• install fabric>=2.5.0 as a dependency

0.4.1 (2020-08-10)

Bug fix

• remove workaround previously needed for Python 2 Fabric on Python 3 projects

0.4.0 (2020-08-10)

Improvements

• add Fabric 2 equivalents of existing features

0.3.2 (2019-12-04)

Improvements

• add optional “push_to_origin” argument to “transfer_files_git”

0.3.1 (2019-07-17)

Bug fix

• use Python 3 compatible syntax for octal numbers

0.3.0 (2019-07-17)

Improvements

• create a separate uWSGI config file, rather than a symbolic link, reducing downtime

0.2.5 (2019-01-17)

Improvements

• Test Nginx config to make errors fail loudly

• Added a TODO file

• Fail deployment if Django issues a warning

• Allow Django “check” fail level to be specified

• Added download_postgres_db, mirror_postgres_db and mirror_media commands

• Push the git branch configured in env.branch

• Remove install dependencies, since you probably already have Fabric 1.x installed

0.2.2 (2016-10-11)

• Add README

• Add license information

• Remove redundant Supervisor, Bazaar and Gunicorn rules