Skip to main content

Detect, classify, and block schema drift in PostgreSQL before your pipelines break

Project description

DriftBrake-Banner

Detect, classify, and block schema drift in PostgreSQL before your pipelines break

Tests PyPI Latest Release PyPI Downloads Python License

DriftBrake is a Python package that automatically reads the current schema of a PostgreSQL database, compares it against a versioned contract, classifies drifts by impact, and can block pipelines before they break in production.

The tool catches bugs that could silently corrupt or break data pipelines before they reach production. The idea is simple: you create a "contract" describing exactly how your database should look. Before running any pipeline, DriftBrake compares the actual database against this contract and warns you (or blocks you) if anything has changed.


Documentation: US-Click here! | BR-Clique Aqui!
Readme versão BR: Clique aqui!
Classification Audit: US-Click here! | BR-Clique Aqui!
Changelog: Click here!

The tool

DriftBrake is not a migration tool. It doesn't apply changes to the database, doesn't generate SQL scripts, and doesn't manage schema versions.

DriftBrake runs before pipelines execute, verifying that the actual database still respects the contract expected by its data consumers. It detects deviations, classifies impact, and blocks execution when necessary — but never alters the database.

Summary:

  • Reads the PostgreSQL schema
  • Compares it against a contract
  • Classifies changes by impact
  • Blocks pipelines on breaking changes
  • Generates JSON, HTML, and Markdown reports

Installation

# Installs the psycopg2-binary driver, required for the postgres connection
pip install "driftbrake[postgres]"

The [dev] extra includes pre-commit, ruff, mypy, pytest, and the other development tools.

Example workflow

The schema.lock.json (the contract) is generated automatically when you run init.

actual database
       │
       ▼
  [1] init          ← takes the "snapshot" of the database and saves it as the contract
       │
       ▼
 schema.lock.json   ← this file is the contract (versioned in Git)
       │
       │    (the database may change over time)
       │
       ▼
  [2] check         ← compares the current database against the contract
       │
       ├── all equal → pipeline can run
       └── change detected → alert or block

When a change is deliberate and approved, use update-contract to update the contract. When you want to compare two states without touching the contract, use diff or snapshot.


Glossary

Contract (schema.lock.json): the JSON file that describes how the database should look. It works as a "lock file" (hence the name) — just as package-lock.json pins package versions, this file pins the database structure.

BREAKING: a change that breaks existing consumers. Examples: removing a column, changing a type from INTEGER to VARCHAR, adding a NOT NULL column without a default.

WARNING: a change that deserves attention but doesn't necessarily break anything right now. Examples: adding a NOT NULL column with a default, changing a default value.

SAFE: a change with no impact on existing consumers. Examples: adding a nullable column, creating a new table.

Diff: the difference found between the contract (what was expected) and the actual database (what exists now).

Expected contract & current database: the comparator always treats the contract as the "agreed-upon truth" and the database as the "actual state." If something exists in the database but not in the contract, it's an addition. If it exists in the contract but is missing from the database, it's a removal.

Initial usage example

Create the initial contract:

driftbrake init --db-url "$DATABASE_URL" --output schema.lock.json

Verify before the pipeline runs:

driftbrake check \
  --db-url "$DATABASE_URL" \
  --contract schema.lock.json \
  --fail-on BREAKING \
  --json schema_diff.json \
  --html schema_report.html \
  --markdown schema_report.md

Update the contract after approving changes:

driftbrake update-contract --db-url "$DATABASE_URL" --contract schema.lock.json

Save a snapshot without changing the contract:

driftbrake snapshot --db-url "$DATABASE_URL" --output snapshots/schema_before.json

Compare two snapshots (or a snapshot against the live database):

driftbrake diff --old snapshots/schema_before.json --new-db "$DATABASE_URL"

How it works

The current flow:

schema.lock.json (contract versioned in Git)
        │
        ▼
DriftBrake connects to PostgreSQL
        │
        ▼
reads the current schema automatically
        │
        ▼
compares expected against current
        │
        ├── OK ──────────────────── pipeline runs
        │
        └── BREAKING ────────────── pipeline blocked
                                    ├── displays in terminal
                                    ├── generates schema_diff.json
                                    └── generates schema_report.html

Change types detected

The tool detects the following categories of change in every comparison:

Type What it means
table_added A new table appeared in the database
table_removed A table that existed is gone from the database
column_added A NOT NULL column was added to an existing table
nullable_column_added A nullable column was added to an existing table
column_removed A column was removed from an existing table
type_changed A column's data type changed (e.g. INTEGERTEXT)
nullable_changed The column stopped accepting NULL or started accepting it
default_changed The column's default value changed or was removed
primary_key_changed A column gained or lost its primary key
unique_changed A UNIQUE constraint was added or removed
foreign_key_changed A foreign key was modified
foreign_key_added A foreign key was created where there was none
ordinal_position_changed The column's position in the table changed
possible_rename A column was removed and a similar one was added in the same table. The tool only flags this as a suspicion of rename, never as a confirmation. Always classified as WARNING.

possible_rename is a heuristic, never a confirmation. DriftBrake flags the suspicion when a removed column and an added column appear type-compatible. Final validation must be done by whoever reviews the migration.


possible_rename confidence

Each possible_rename occurrence carries a confidence field indicating how certain the heuristic is:

Level Criteria
high Similar name + same type + close ordinal position (difference ≤ 2)
medium Same type + close ordinal position (difference ≤ 2)
low Only type-compatible (SAFE or WARNING in the type matrix)

Important rules:

  • possible_rename is never automatically classified as BREAKING — always WARNING.
  • A confidence: "high" is still a suspicion, not a certainty.
  • Always review migrations before accepting a rename with driftbrake update-contract.

PostgreSQL types (compatibility matrix)

Conversion Severity
varchar(50)varchar(100) SAFE
varchar(100)varchar(50) BREAKING
textvarchar(n) BREAKING
varchar(n)text SAFE
integerbigint WARNING
bigintinteger BREAKING
smallintinteger SAFE
numeric(10,2)numeric(12,2) SAFE
numeric(12,2)numeric(10,2) BREAKING
numerictext BREAKING
datetimestamp WARNING
timestampdate BREAKING


Python library

The same detection engine is available as a Python library. The simplest integration is a single line before the pipeline:

from driftbrake import DriftBrake

DriftBrake.run_from_env()

run_pipeline()

For pipelines that need to inspect the result before acting, protect() returns a DiffResult and raises typed exceptions on failure:

from driftbrake import DriftBrake
from driftbrake.exceptions import BreakingChangesDetected

try:
    result = DriftBrake.from_env().protect()
except BreakingChangesDetected as e:
    notify_slack(f"Pipeline blocked: {len(e.result.changes)} breaking changes")
    raise

run_pipeline(result)

See the full documentation for policy overrides, custom reporters, async support, and the context manager API.

Stack

  • SQLAlchemy — PostgreSQL reflection/inspection
  • Typer — CLI
  • Rich — terminal output
  • Jinja2 — HTML templates
  • python-dotenv — environment variables
  • PyYAML — configuration
  • pytest — tests

License

MIT license

Author

Yuri Pontes — Former Cabo (Corporal equivalent) - Brazilian Army, transitioning to data engineering.

LinkedIn · GitHub

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

driftbrake-0.1.1.tar.gz (36.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

driftbrake-0.1.1-py3-none-any.whl (50.5 kB view details)

Uploaded Python 3

File details

Details for the file driftbrake-0.1.1.tar.gz.

File metadata

  • Download URL: driftbrake-0.1.1.tar.gz
  • Upload date:
  • Size: 36.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.5

File hashes

Hashes for driftbrake-0.1.1.tar.gz
Algorithm Hash digest
SHA256 d2f5921ff83debeae4c74a1d2d5603ef56048fe9e265c5ba72927d7b2b010ca3
MD5 cf3c1090c5ae516bd39197072a792019
BLAKE2b-256 254ef0f0b60cfd0ac512969799ef21a9abc68f47ba51c5064edf41cd1a291973

See more details on using hashes here.

File details

Details for the file driftbrake-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: driftbrake-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 50.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.5

File hashes

Hashes for driftbrake-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 8146ad4ae30088b6977b105306d73f61c87674b2cc86ac595fcf1e88f4623dbe
MD5 6327ceccaa02e938d116473fd22ff5a6
BLAKE2b-256 2f66c64742f765728e3f0d465e3e7ea90efa982f78682677e4539137aa1cbfa4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page