Skip to main content

bricksentry

PyPI Python CI License

Change-safety for Databricks pipelines and the AI agents that edit them.

Website and docs: bricksentry.org

Try it in 30 seconds (no Databricks needed)

pip install "bricksentry[duckdb]"
bricksentry demo

An AI agent changes a 20,000-row revenue table. bricksentry previews its DELETE on a clone (production untouched), blocks its pull request for a join that duplicates 40 orders, and routes the fixed version to the owners of a certified table downstream. Everything runs locally on DuckDB.

bricksentry demo: an agent's change is previewed, blocked for duplicate keys, then routed to the table owners

What it does

Before a change reaches production, bricksentry answers three questions, in a pull-request comment for humans and in structured JSON for agents:

  1. What does it do to the data? A key-based data diff: added, removed and changed rows, schema changes, per-column changes, numeric totals, duplicate keys, and where the change is concentrated ("100% of affected rows are region = APAC").
  2. Who gets hurt? The downstream blast radius from Unity Catalog lineage: tables, jobs, notebooks, Lakeflow pipelines, dashboards and queries, with owners and tags.
  3. Is it safe, and how do I undo it? A deterministic policy verdict (pass · warn · needs_human · block) and the exact RESTORE TABLE … VERSION AS OF n rollback.

For your Databricks workspace:

pip install "bricksentry[databricks,mcp]"
bricksentry init        # writes bricksentry.yml and policy.yml

Why it exists

AI now writes much of our pipeline code, and checking what that code does to the data hasn't kept up. Code review can't see a silent 3% revenue shift or a join that fans out. The tools that do catch this mostly cover dbt, or need a paid platform. Most Databricks work is PySpark notebooks, Jobs and Lakeflow pipelines, and bricksentry is built for that work:

  • Unity Catalog runtime lineage, not SQL parsing, so PySpark notebooks and jobs are included.
  • SQL pushdown: all diffing runs on your SQL warehouse, so no Spark session is needed and calls stay cheap enough for an agent loop.
  • One engine, three faces: CLI and GitHub Action · Python API · MCP server.

For AI agents: the MCP server

bricksentry mcp --config bricksentry.yml                 # stdio
claude mcp add bricksentry -- bricksentry mcp --config /abs/path/bricksentry.yml
Tool What the agent gets
whats_downstream(table, depth) Downstream assets with owners and tags. Call it before editing.
rollback_plan(table) The restore SQL. Capture it before changing anything.
preview_write(sql, target, keys?) Runs one write against a disposable shallow clone, never production, and returns the diff plus verdict.
assess_change(tables, candidates?, pr?) Diffs prod vs. the candidate built by your change, plus lineage and policy. Returns a verdict and an id.
explain_diff(assessment_id or table) Plain-English summary.
safe_to_ship(assessment_id) safe_to_ship, the reasons, and an explicit next_step to follow.

Designed for agents:

  • The verdict never comes from an LLM. Rules in policy.yml decide, so the same change always gets the same answer.
  • Read-only by default. Only preview_write writes, and only to a sandbox clone. It accepts a single INSERT / MERGE / UPDATE / DELETE / CREATE OR REPLACE whose target must be the clone, and it refuses DROP, TRUNCATE, ALTER, GRANT and multiple statements.
  • Errors come back as {"error": ...}, so agents can recover instead of crashing.
  • Bounded cost: deterministic hash sampling (sample_pct), capped samples, and lineage limited by depth and lookback window.

The same tools are plain Python functions in bricksentry.agent_tools, so you can use them from LangGraph, the OpenAI Agents SDK or the Databricks Agent Framework without MCP.

To watch the whole loop, run bricksentry demo (see the top of this page). bricksentry demo --report report.md also writes the pull-request comment a reviewer would see.

For data engineers: CLI and CI

bricksentry init                                        # starter bricksentry.yml + policy.yml
bricksentry impact main.sales.orders                    # blast radius
bricksentry diff main.sales.orders main.sales_pr_42.orders
bricksentry preview main.sales.orders --sql "MERGE INTO {{target}} t USING updates u ON t.order_id = u.order_id WHEN MATCHED THEN UPDATE SET *"
bricksentry assess --from-git origin/main --pr 42 --format markdown --fail-on needs_human
bricksentry rollback main.sales.orders

Exit codes: 0 below --fail-on, 1 at or above it, 2 for configuration errors.

In GitHub Actions. Build your PR's version of the pipeline into a per-PR schema (for example with a Databricks Asset Bundle ci target), then add:

- run: pip install "bricksentry[databricks]"
- run: >
    bricksentry assess
    --from-git origin/${{ github.base_ref }}
    --pr ${{ github.event.pull_request.number }}
    --comment --output bricksentry-report.md
    --fail-on needs_human
  env:
    GITHUB_TOKEN: ${{ github.token }}
    DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}
    DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
    DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
    DATABRICKS_WAREHOUSE_ID: ${{ secrets.DATABRICKS_WAREHOUSE_ID }}

It detects changed tables from the files in the PR and posts or updates one PR comment, like this one for the demo's fan-out bug:

A bricksentry pull-request comment: blocked for 40 duplicate keys, with the data diff and blast radius

Check out with fetch-depth: 0 and give the job pull-requests: write permission.

Or use the bundled GitHub Action, which also writes the report to the job summary:

- uses: bricksentry/bricksentry@v0.1.0
  with:
    config: bricksentry.yml
    fail-on: needs_human

Python API

from bricksentry import Sentry

s = Sentry()                                   # reads ./bricksentry.yml
a = s.assess(["main.sales.orders"], pr="42")
print(a.verdict, a.notify)
print(a.tables[0].summary)

s.preview_write("DELETE FROM main.sales.orders WHERE status = 'test'", "main.sales.orders")

Configuration

backend: {type: databricks, warehouse_id: abc123}      # auth: standard Databricks SDK chain
sandbox_schema: main.bricksentry_sandbox
candidate_pattern: "{catalog}.{schema}_pr_{pr}.{table}"
sample_pct: 10
lineage: {type: databricks, depth: 3, lookback_days: 90}
tables:
  main.sales.orders:
    keys: [order_id]
    segment_by: [region]
    paths: ["src/pipelines/orders/**", "resources/orders_job.yml"]
policy: policy.yml

Policy conditions

table (glob) · tag · downstream_tag · downstream_kind · downstream_count_gt · removed_pct_gt · added_pct_gt · changed_pct_gt · row_count_change_pct_gt · row_count_drop_pct_gt · column_removed · column_added · column_type_changed · duplicate_keys · sum_change_pct_gt / null_increase_gt (optionally with column) · any_change · diff_error

All conditions in a when must hold. The most severe matching rule sets the verdict. Unknown conditions are rejected at load time, so a typo can't silently disable a rule.

Permissions needed on Databricks

  • CAN USE on a SQL warehouse.
  • SELECT on the tables you assess and their candidates.
  • SELECT on system.access.table_lineage, plus system.information_schema for owners and tags. Without these, bricksentry still diffs; lineage is just empty.
  • CREATE TABLE on sandbox_schema, for preview_write.

Known limits (v0.1.x)

  • Lineage is recorded at runtime. Brand-new tables have no downstream history yet. Unity Catalog doesn't capture RDD operations, and UDFs hide column-level mapping.
  • Keys are required. Tables without a stable key can't be diffed row by row yet (an aggregate-profile mode is planned).
  • Changed-table detection uses path globs from config. It doesn't parse Asset Bundles yet.
  • The Databricks backend is tested against a mocked SDK. Run it against a dev workspace before relying on it in CI.

Roadmap

Asset Bundle parsing for automatic table detection · column-level blast radius from system.access.column_lineage · an aggregate diff for keyless tables · Lakeflow expectations in the verdict · bricksentry watch for scheduled drift checks · a Databricks App UI.

Contributing

Bug reports from real pipelines help the most. See CONTRIBUTING.md, and SECURITY.md for reporting vulnerabilities privately.

License

Apache-2.0. Created and maintained by Brahma.

Metadata

Release files for bricksentry 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bricksentry 0.1.1
File Size Uploaded Provenance Actions
bricksentry-0.1.1.tar.gz 44.1 kB Without provenance Details for bricksentry-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for bricksentry 0.1.1
File Interpreter ABI Platform Actions
bricksentry-0.1.1-py3-none-any.whl Python 3 none any Details for bricksentry-0.1.1-py3-none-any.whl

Total release size: 90.0 kB

Release files / bricksentry-0.1.1.tar.gz

Download URL bricksentry-0.1.1.tar.gz
Size 44.1 kB
Tags Source
SHA-256 checksum
How to use checksums
99afb8969e7cf66c9cd9ae82e2bc72cc86d219a66d61d161166986d5660edf87
BLAKE2b-256 checksum
How to use checksums
74bc8520fb34827e8b6a76a75913ae1a09039467ef78fe8a7ab2ca020516a1b1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.2

Release files / bricksentry-0.1.1-py3-none-any.whl

Download URL bricksentry-0.1.1-py3-none-any.whl
Size 45.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e03844e569196d7b0bdf6897d9de5e74152f07600fdd5f275d0953a1605e3ec6
BLAKE2b-256 checksum
How to use checksums
a1d2d8cb74bef76fbaf2c76930dde290af9ca3df85a3190c11071ebdde63b7d2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.2

Security for bricksentry 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page