DBS — Django Backup Solution

🚧 Under development. DBS is in active development and its API may change.

Created by Sudum Technology — Research and Development sector. Our very tiny contribution to this world.

A backup library you drop into a Django project's source. It reads your models, relations and files and writes one encrypted file that is redundant and self-healing: every backup stores two copies of the data plus Reed-Solomon parity, so silent corruption — the kind a non-ECC RAM stick produces — is detected and repaired on restore instead of quietly poisoning your data.

io  ──►  security (Argon2id + AES-256-GCM)  ──►  integrity (2 copies · per-block
hashes · Reed-Solomon)  ──►  data (models · relations · files)

Features

• Two self-healing copies in one file plus Reed-Solomon parity for bit-rot.
• Per-field mapping: value, embedded file, or file-path.
• Passphrase-derived key, never stored (Argon2id).
• Per-file and per-block integrity hashes.
• Scheduled backups on the server, with retention.
• A dbs-client command that pulls backups off your server over SSH.
• CLI · admin-UI download · SFTP.

Install

pip install django-dbs              # on the server: the library + management commands
pip install "django-dbs[client]"    # on your machine: the dbs-client command

Add the app on the server:

INSTALLED_APPS = [..., "dbs"]

The [client] extra adds paramiko (SSH/SFTP) and, on Python 3.9/3.10, a TOML parser. [ssh] remains available as the server-side SFTP extra.

The [client] extra still installs Django, because DBS depends on it. You do not need a Django project on the machine running dbs-client — the client never reads Django settings.

On the server

1. (Optional) declare what to back up

By default DBS auto-discovers every model, treats FileField/ImageField as files, and preserves relations. Register a model only to override the defaults — in a dbs.py module inside your app (auto-discovered like admin.py):

# myapp/dbs.py
from dbs import backup_registry, FieldType, ModelBackup
from .models import Invoice

@backup_registry.register(Invoice)
class InvoiceBackup(ModelBackup):
    overrides = {
        "scanned_pdf_path": FieldType.FILE_PATH,  # CharField holding a path -> embed the file
        "render_cache": FieldType.EXCLUDE,        # don't back this column up
    }
    file_roots = ["/srv/myapp/uploads"]           # extra non-model file trees

FieldType values: VALUE (default), FILE (embed a FileField's bytes), FILE_PATH (a string column whose path's file is embedded), EXCLUDE.

2. Back up / restore / validate from the CLI

python manage.py dbs_backup   backup.dbs            # prompts for a passphrase
python manage.py dbs_validate backup.dbs            # structural check, no passphrase
python manage.py dbs_validate backup.dbs --passphrase secret  # + verify decryption
python manage.py dbs_restore  backup.dbs            # restore rows + files

The passphrase comes from --passphrase, then $DBS_PASSPHRASE, then a prompt. --passphrase-stdin reads it from the first line of standard input instead, which keeps it out of the process's command line:

printf '%s\n' "$SECRET" | python manage.py dbs_backup backup.dbs --passphrase-stdin

Signals during restore

dbs_restore saves each row the same way loaddata does: with raw=True. A pre_save/post_save receiver that re-runs business side effects — awarding points, sending notifications, recomputing derived state — must return early on raw saves, or a restore replays those effects against a half-loaded database:

@receiver(post_save, sender=Order)
def on_order_saved(sender, instance, created, **kwargs):
    if kwargs.get("raw"):
        return
    ...

3. Automatic backups on a schedule

dbs_schedule runs backups on a repeating interval, prunes old ones, and can push each backup to another host in the same cycle.

export DBS_PASSPHRASE='…'
python manage.py dbs_schedule --interval 6h --output-dir /var/backups/myproject --keep 14

It reads the passphrase from $DBS_PASSPHRASE and never prompts; if the variable is missing it exits immediately with an explanation rather than hanging on an invisible prompt. A cycle that fails is logged and the schedule continues.

Backups are named prefix-YYYYMMDD-HHMMSSZ.dbs in UTC, so sorting by name is sorting by time. Retention only ever considers files matching that convention and the configured prefix — anything else in the directory is left alone, and --keep must be at least 1.

systemd:

[Unit]
Description=DBS scheduled backups
After=network-online.target

[Service]
User=deploy
WorkingDirectory=/srv/myproject
Environment=DBS_PASSPHRASE=…
ExecStart=/srv/myproject/.venv/bin/python manage.py dbs_schedule --interval 6h \
          --output-dir /var/backups/myproject --keep 14
Restart=always

[Install]
WantedBy=multi-user.target

SIGTERM stops the loop straight away rather than after the current interval, so systemctl restart and container shutdowns are immediate. Run one scheduler per output directory — two loops sharing a directory will race on retention. Prefer Environment= in a chmod 600 unit file, or a systemd credential, over a passphrase in a shell profile.

cron / Docker: use --once from cron for a single cycle, or run the loop as the container's main process.

4. Download / upload from the admin UI

# urls.py
urlpatterns += [path("dbs/", include("dbs.contrib.urls"))]

Superusers can then visit /dbs/backup/ to download an encrypted backup and /dbs/restore/ to upload one. The passphrase is entered in the form and never stored server-side.

5. Ship a backup to another server (SFTP)

# settings.py — profiles reference a key by path; no secrets embedded
DBS_SSH_TARGETS = {
    "offsite": {
        "host": "backups.example.com", "username": "deploy",
        "key_filename": "/home/deploy/.ssh/id_ed25519",
        "remote_dir": "/var/backups/myproject",
        "known_hosts": "/home/deploy/.ssh/known_hosts",
    }
}

from dbs import create_backup
from dbs.transports import SSHTarget, push_backup

data = create_backup("my passphrase")
push_backup(data, "backup.dbs", SSHTarget.from_settings("offsite"))

On your machine: dbs-client

dbs-client connects to a server where django-dbs is installed, asks it to make a fresh backup, and downloads the file — all over one encrypted SSH connection. It can also list, pull, push, prune, and run on a schedule of its own.

pip install "django-dbs[client]"
dbs-client init                 # writes dbs-client.toml, mode 600
$EDITOR dbs-client.toml
dbs-client test-connection
dbs-client backup

Configuration

dbs-client reads the first config it finds: --config PATH, then $DBS_CLIENT_CONFIG, then ./dbs-client.toml, then ~/.config/dbs/client.toml. dbs-client init --print writes an annotated template to standard output.

[defaults]
server = "production"
dest = "~/dbs-backups"
keep = 14
keep_remote = 7
interval = "6h"

[servers.production]
host = "app.example.com"
username = "deploy"
port = 22
key_filename = "~/.ssh/production.pem"
known_hosts = "~/.ssh/known_hosts"

project_dir = "/srv/myproject"
python = "/srv/myproject/.venv/bin/python"
manage = "manage.py"
django_settings_module = "myproject.settings.production"
remote_dir = "/var/backups/myproject"

passphrase_env = "DBS_PASSPHRASE"
prefix = "production"
keep = 30

Anything under [defaults] applies to every server unless that server overrides it. Pick a server with --server NAME, before or after the subcommand.

A key that isn't recognised is an error, not a warning — a typo like known_host would otherwise silently disable host key checking.

If the file holds a literal password, passphrase or key_passphrase and is readable by other users, dbs-client refuses to run and tells you to chmod 600 it. A config writable by other users is refused regardless, because it decides which command runs on your server.

Commands

dbs-client test-connection
# [ok] ssh authentication      deploy@app.example.com:22
# [ok] host key policy         reject-unknown
# [ok] remote shell            Linux 6.1.0
# [ok] remote directory        /var/backups/myproject
# [ok] django-dbs on server    0.2.0
# [ok] manage.py dbs_backup    ready
# [ok] backups present         7

dbs-client backup --keep 14 --keep-remote 7
dbs-client pull --latest
dbs-client prune --keep 7 --local-only --dry-run
dbs-client schedule --interval 6h --keep 30

backup triggers manage.py dbs_backup on the server, downloads the result to dest, validates it locally, then applies retention. --no-fetch leaves it on the server, --delete-remote removes the remote copy once it is safely downloaded, and --no-validate skips the local check. Downloads stream to a .part file and are renamed into place only once complete, so an interrupted transfer can never be mistaken for a good backup.

schedule uses the same loop as the server-side command: SIGTERM and Ctrl-C stop it promptly, and a failing cycle is logged without ending the schedule. It resolves the passphrase once at start-up and never prompts.

validate needs no Django project and no settings — it works anywhere the file is.

SSH authentication

Any secret can be given directly or read from an environment variable by adding _env to the key name, which keeps it out of the file entirely.

Host keys are verified and unknown hosts are rejected by default. Point known_hosts at a file (~/.ssh/known_hosts is typical) or leave it unset to use the system host keys. auto_add_host_key = true trusts whatever key the server presents on first contact — test-connection always prints which policy is in effect so this doesn't get left on by accident.

Passphrases in unattended runs

The backup passphrase never appears in a command line — not locally, and not on the server. There is no --passphrase flag on dbs-client at all, because process arguments are readable by every user on the machine via /proc.

• passphrase_transport = "stdin" (default, needs django-dbs ≥ 0.2.0 on the server). The client runs manage.py dbs_backup --passphrase-stdin and writes the passphrase down the encrypted SSH channel's standard input. It is in neither the remote process's arguments nor its environment, and never touches the server's disk.
• passphrase_transport = "env" (for servers still on 0.1.x). The passphrase is still sent over stdin, but a shell reads it into DBS_PASSPHRASE for the backup process. It is then visible in that process's environment to root and to the same user — the same exposure as a systemd Environment= line, and better than a command-line argument.

Locally the passphrase is resolved from passphrase_env, then a literal passphrase, then $DBS_PASSPHRASE, then an interactive prompt.

Python API

from dbs import create_backup, restore_backup, validate_backup

blob = create_backup("passphrase", output="backup.dbs")
report = validate_backup(blob, "passphrase")   # report.ok / report.summary()
result = restore_backup(blob, "passphrase")    # result.healed is True if it repaired corruption

from dbs.transports import SSHTarget, open_session

with open_session(SSHTarget.from_settings("offsite")) as session:
    session.push("backup.dbs", "backup.dbs")
    for item in session.details():
        print(item.name, item.size, item.modified)
    session.delete("old-backup.dbs")

open_session reuses one connection for a whole trigger-fetch-prune cycle. The module-level push_backup, pull_backup, pull_backup_to, list_backups, list_backup_details, delete_backup and check_connection helpers open and close a connection each.

from dbs.client import load_client_config, resolve_client_passphrase, backup_and_fetch

profile = load_client_config().server("production")
result = backup_and_fetch(profile, resolve_client_passphrase(profile), keep=14)
print(result.local_path, result.size)

How it heals

On write, the encrypted stream is split into blocks; each block gets a BLAKE2b hash and a layer of Reed-Solomon parity, and the whole stream is stored twice (plus the header and manifest are stored twice). On read, each block is taken from whichever copy verifies; sparse bit-flips are corrected in place by Reed-Solomon even when both copies are hit. Every recovered block is checked against its stored hash, so a mis-correction can never slip through — and a freshly written backup is re-read and verified end-to-end (verify-after-write) before the command reports success.

What it can recover from: whole-block loss in one copy, and sparse byte errors (up to the parity budget, ~8 bytes per 255-byte codeword by default) in both copies. What it cannot: a block destroyed beyond the parity budget in both copies — DBS then refuses to restore and reports exactly which blocks failed, rather than producing silently wrong data.

Security model

• Argon2id derives a key from your passphrase (memory-hard ⇒ brute-force resistant). Raise KDFParams cost for more resistance.
• Envelope encryption: a random data key encrypts the payload with AES-256-GCM; that data key is wrapped by the passphrase-derived key. The file stores only the salt, Argon2 parameters and the wrapped key — never the passphrase and never the raw data key.
• A wrong passphrase fails the GCM tag check and is reported as such; it can never yield partial/garbage data.
• Transfers run over SSH/SFTP with host key verification on by default.
• Passphrases are never passed as command-line arguments; see Passphrases in unattended runs above.

Restore semantics

• The database load runs in a single transaction: if anything fails midway, every row is rolled back and the database is left untouched.
• Backup collection also runs in a transaction, so a backup is a consistent snapshot even while the application keeps writing.
• Writing a .dbs file is atomic and durable (temp file + fsync + rename), and verify-after-write re-reads the bytes that actually landed on disk. Uploads and downloads land the same way, via a .part file that is renamed on success.
• File restores are confined: FILE_PATH and file-root entries are only written when the target resolves under a directory listed in DBS_RESTORE_ROOTS (or, if unset, DBS_FILE_ROOTS). Anything else raises RestoreError. Configure the allow-list before restoring backups that embed external files.
• Restoring merges by primary key into the target database; rows created after the backup was taken are not deleted.
• dbs-client never restores into a server's database. Use push to place a file there, then run dbs_restore on the server deliberately.

Observability

DBS logs to the dbs logger: backup/restore start and finish, skipped source files, SFTP transfers, scheduled cycles and retention, and — most importantly — a WARNING whenever silent corruption was detected and healed. Route that logger to your monitoring so a degrading disk is noticed before both copies are damaged:

LOGGING = {
    "version": 1,
    "loggers": {"dbs": {"handlers": ["console"], "level": "INFO"}},
}

dbs-client --verbose prints the same log to stderr. Files that could not be read during a backup are also recorded in the manifest (validate_backup(...).stats["skipped_files"]).

Settings reference

Development

pip install -e ".[dev]"
pytest

License

MIT