Skip to main content

bricksentry

PyPI Python CI License

Change-safety for Databricks pipelines and the AI agents that edit them.

Website and docs: bricksentry.org

Try it in 30 seconds (no Databricks needed)

pip install "bricksentry[duckdb]"
bricksentry demo

An AI agent changes a 20,000-row revenue table. bricksentry previews its DELETE on a clone (production untouched), blocks its pull request for a join that duplicates 40 orders, and routes the fixed version to the owners of a certified table downstream. Everything runs locally on DuckDB.

bricksentry demo: an agent's change is previewed, blocked for duplicate keys, then routed to the table owners

What it does

Before a change reaches production, bricksentry answers three questions, in a pull-request comment for humans and in structured JSON for agents:

  1. What does it do to the data? A key-based data diff: added, removed and changed rows, schema changes, per-column changes, numeric totals, duplicate keys, and where the change is concentrated ("100% of affected rows are region = APAC").
  2. Who gets hurt? The downstream blast radius from Unity Catalog lineage: tables, jobs, notebooks, Lakeflow pipelines, dashboards and queries, with owners and tags.
  3. Is it safe, and how do I undo it? A deterministic policy verdict (pass · warn · needs_human · block) and the exact RESTORE TABLE … VERSION AS OF n rollback.

For your Databricks workspace:

pip install "bricksentry[databricks,mcp]"
bricksentry init        # writes bricksentry.yml and policy.yml

Why it exists

AI now writes much of our pipeline code, and checking what that code does to the data hasn't kept up. Code review can't see a silent 3% revenue shift or a join that fans out. The tools that do catch this mostly cover dbt, or need a paid platform. Most Databricks work is PySpark notebooks, Jobs and Lakeflow pipelines, and bricksentry is built for that work:

  • Unity Catalog runtime lineage, not SQL parsing, so PySpark notebooks and jobs are included.
  • SQL pushdown: all diffing runs on your SQL warehouse, so no Spark session is needed and calls stay cheap enough for an agent loop.
  • One engine, three faces: CLI and GitHub Action · Python API · MCP server.

For AI agents: the MCP server

bricksentry mcp --config bricksentry.yml                 # stdio
claude mcp add bricksentry -- bricksentry mcp --config /abs/path/bricksentry.yml
Tool What the agent gets
whats_downstream(table, depth) Downstream assets with owners and tags. Call it before editing.
rollback_plan(table) The restore SQL. Capture it before changing anything.
preview_write(sql, target, keys?) Runs one write against a disposable shallow clone, never production, and returns the diff plus verdict.
assess_change(tables, candidates?, pr?) Diffs prod vs. the candidate built by your change, plus lineage and policy. Returns a verdict and an id.
explain_diff(assessment_id or table) Plain-English summary.
safe_to_ship(assessment_id) safe_to_ship, the reasons, and an explicit next_step to follow.

Designed for agents:

  • The verdict never comes from an LLM. Rules in policy.yml decide, so the same change always gets the same answer.
  • Read-only by default. Only preview_write writes, and only to a sandbox clone. It accepts a single INSERT / MERGE / UPDATE / DELETE / CREATE OR REPLACE whose target must be the clone, and it refuses DROP, TRUNCATE, ALTER, GRANT and multiple statements.
  • Errors come back as {"error": ...}, so agents can recover instead of crashing.
  • Bounded cost: deterministic hash sampling (sample_pct), capped samples, and lineage limited by depth and lookback window.

The same tools are plain Python functions in bricksentry.agent_tools, so you can use them from LangGraph, the OpenAI Agents SDK or the Databricks Agent Framework without MCP.

To watch the whole loop, run bricksentry demo (see the top of this page). bricksentry demo --report report.md also writes the pull-request comment a reviewer would see.

For data engineers: CLI and CI

bricksentry init                                        # starter bricksentry.yml + policy.yml
bricksentry impact main.sales.orders                    # blast radius
bricksentry diff main.sales.orders main.sales_pr_42.orders
bricksentry preview main.sales.orders --sql "MERGE INTO {{target}} t USING updates u ON t.order_id = u.order_id WHEN MATCHED THEN UPDATE SET *"
bricksentry assess --from-git origin/main --pr 42 --format markdown --fail-on needs_human
bricksentry rollback main.sales.orders

Exit codes: 0 below --fail-on, 1 at or above it, 2 for configuration errors.

In GitHub Actions. Build your PR's version of the pipeline into a per-PR schema (for example with a Databricks Asset Bundle ci target), then add:

- run: pip install "bricksentry[databricks]"
- run: >
    bricksentry assess
    --from-git origin/${{ github.base_ref }}
    --pr ${{ github.event.pull_request.number }}
    --comment --output bricksentry-report.md
    --fail-on needs_human
  env:
    GITHUB_TOKEN: ${{ github.token }}
    DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}
    DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
    DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
    DATABRICKS_WAREHOUSE_ID: ${{ secrets.DATABRICKS_WAREHOUSE_ID }}

It detects changed tables from the files in the PR and posts or updates one PR comment, like this one for the demo's fan-out bug:

A bricksentry pull-request comment: blocked for 40 duplicate keys, with the data diff and blast radius

Check out with fetch-depth: 0 and give the job pull-requests: write permission.

Or use the bundled GitHub Action, which also writes the report to the job summary:

- uses: bricksentry/bricksentry@v0.1.0
  with:
    config: bricksentry.yml
    fail-on: needs_human

Python API

from bricksentry import Sentry

s = Sentry()                                   # reads ./bricksentry.yml
a = s.assess(["main.sales.orders"], pr="42")
print(a.verdict, a.notify)
print(a.tables[0].summary)

s.preview_write("DELETE FROM main.sales.orders WHERE status = 'test'", "main.sales.orders")

Configuration

backend: {type: databricks, warehouse_id: abc123}      # auth: standard Databricks SDK chain
sandbox_schema: main.bricksentry_sandbox
candidate_pattern: "{catalog}.{schema}_pr_{pr}.{table}"
sample_pct: 10
lineage: {type: databricks, depth: 3, lookback_days: 90}
tables:
  main.sales.orders:
    keys: [order_id]
    segment_by: [region]
    paths: ["src/pipelines/orders/**", "resources/orders_job.yml"]
policy: policy.yml

Policy conditions

table (glob) · tag · downstream_tag · downstream_kind · downstream_count_gt · removed_pct_gt · added_pct_gt · changed_pct_gt · row_count_change_pct_gt · row_count_drop_pct_gt · column_removed · column_added · column_type_changed · duplicate_keys · null_keys · sum_change_pct_gt / null_increase_gt (optionally with column) · any_change · diff_error

All conditions in a when must hold. The most severe matching rule sets the verdict. Unknown conditions are rejected at load time, so a typo can't silently disable a rule.

Permissions needed on Databricks

  • CAN USE on a SQL warehouse.
  • SELECT on the tables you assess and their candidates.
  • SELECT on system.access.table_lineage, plus system.information_schema for owners and tags. Without these, bricksentry still diffs; lineage is just empty.
  • CREATE TABLE on sandbox_schema, for preview_write.

Known limits (v0.1.x)

  • Lineage is recorded at runtime. Brand-new tables have no downstream history yet. Unity Catalog doesn't capture RDD operations, and UDFs hide column-level mapping.
  • Keys are required. Tables without a stable key can't be diffed row by row yet (an aggregate-profile mode is planned).
  • Changed-table detection uses path globs from config. It doesn't parse Asset Bundles yet.
  • The Databricks backend is tested against a mocked SDK. Run it against a dev workspace before relying on it in CI.

Roadmap

Asset Bundle parsing for automatic table detection · column-level blast radius from system.access.column_lineage · an aggregate diff for keyless tables · Lakeflow expectations in the verdict · bricksentry watch for scheduled drift checks · a Databricks App UI.

Contributing

Bug reports from real pipelines help the most. See CONTRIBUTING.md, and SECURITY.md for reporting vulnerabilities privately.

License

Apache-2.0. Created and maintained by Brahma.

Metadata

Release files for bricksentry 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bricksentry 0.1.2
File Size Uploaded Provenance Actions
bricksentry-0.1.2.tar.gz 45.1 kB Without provenance Details for bricksentry-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for bricksentry 0.1.2
File Interpreter ABI Platform Actions
bricksentry-0.1.2-py3-none-any.whl Python 3 none any Details for bricksentry-0.1.2-py3-none-any.whl

Total release size: 91.2 kB

Release files / bricksentry-0.1.2.tar.gz

Download URL bricksentry-0.1.2.tar.gz
Size 45.1 kB
Tags Source
SHA-256 checksum
How to use checksums
0ba279f8a777fc1814686a85713bdba416c4db398bd209ff19f80c1daf7f20f3
BLAKE2b-256 checksum
How to use checksums
a61507ad23bce6e747a2be1204a66ac94c317af8fd0f712a6d379fcebdce9993
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.2

Release files / bricksentry-0.1.2-py3-none-any.whl

Download URL bricksentry-0.1.2-py3-none-any.whl
Size 46.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6c7091b4c2e4102057d9702d61e9fe45fbf5beeef0b32f455aab09a941feddbe
BLAKE2b-256 checksum
How to use checksums
acfaf0a843dc0dcab7f19006d75914224fed75dc80fd2bc5442b37e2e087940c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.2

Security for bricksentry 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.2 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page