Veridelta

Veridelta compares two datasets on their keys and reports exactly what changed after applying the variance you declared as expected. It is built for system modernizations, model retrains, and pipeline migrations — anywhere "equal" has to be defined, not assumed.

Why

Deterministic verdicts. Nine fixed transform stages. The same rule produces the same result locally and in a warehouse, verified by a differential harness.
Scale. Lazy Polars scans. Warehouse pushdown compiles comparison SQL and never extracts full tables.
Exactness. Nothing is forgiven unless a rule says so. strict_types treats type drift as a mismatch, not a cast.
CI/CD. Exit codes 0 (match), 1 (drift or a failure), and 2 (invalid arguments). --json on stdout. --html writes a standalone report, --markdown a summary for pull requests, and --otel OpenTelemetry metrics for a dashboard. Artifacts for added, removed, and changed rows.
Schema evolution. schema_mode is intersection, exact, allow_additions, or allow_removals.
Connectors. Snowflake, Databricks, BigQuery, and opt-in Postgres SQL pushdown; Delta Lake and Iceberg scans; PostgreSQL, MySQL, SQL Server, Oracle, SQLite, and more through ConnectorX. Optional extras.

Install

uv add veridelta
# or: pip install veridelta
uv add 'veridelta[snowflake]'   # extras: snowflake, databricks, bigquery, delta, iceberg, database, excel, fuzzy, all

Routing, YAML fields, and time travel: configuration guide.

Architecture

flowchart LR
  subgraph sources [Sources]
    files[Files]
    lakehouse[Delta Iceberg]
    databases[Postgres MySQL SQLite]
    warehouse[Snowflake Databricks BigQuery]
  end
  files --> loader[LoaderFactory]
  lakehouse --> loader
  databases --> loader
  warehouse --> compiler[SQLPushdownCompiler]
  databases -. pushdown .-> compiler
  loader --> engine["DiffEngine"]
  engine --> result[DiffResult]
  compiler --> warehouseSql[Warehouse SQL]
  warehouseSql --> result
  result --> artifacts[Artifacts]
  result --> reports["HTML JSON"]
  result --> exitCode[Exit code]

File, lakehouse, and database sources load through LoaderFactory into a local DiffEngine run. Same-warehouse pairs, and two Postgres tables that set pushdown, compile to SQL and execute in place. Both paths return a DiffResult.

Quick start

Python — DiffEngine consumes LazyFrames:

import polars as pl
from veridelta import DiffConfig, DiffEngine, DiffRule

result = DiffEngine(
    DiffConfig(
        primary_keys=["user_id"],
        rules=[DiffRule(pattern="^AMT_.*", absolute_tolerance=0.05)],
    ),
    pl.scan_parquet("legacy.parquet"),
    pl.scan_parquet("modern.parquet"),
).run()

if not result.summary.is_match:
    raise SystemExit(f"{result.summary.changed_count} rows differ")

YAML — the same comparison for CI:

# veridelta.yaml
primary_keys: ["transaction_id"]
source:
  path: "legacy.parquet"
  format: "parquet"
target:
  path: "modern.parquet"
  format: "parquet"
rules:
  - column_names: ["grand_total"]
    relative_tolerance: 0.01
  - column_names: ["contact_number"]
    regex_replace: {"[^0-9]": ""}

veridelta validate -c veridelta.yaml   # what would stop a run, without reading any rows
veridelta run -c veridelta.yaml

Where next

1. Core Concepts — Python API, DiffResult, rules.
2. YAML and CLI — pipeline automation, --json, artifacts.
3. Advanced Rules — drift resolution on real data.
4. HTML Reports — audit and compliance hand-off.
5. Validate and CI — a database source, veridelta validate, and the GitHub Action.
CI integrations — GitHub Action and GitLab CI template that comment on pull requests.
Configuration — fields, formats, extras, warehouse, lakehouse, and database routing.
API Reference — public Python surface.
Roadmap — work that is not built yet.

License

Apache 2.0.

Metadata

Release files for veridelta 0.12.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for veridelta 0.12.0
File	Size	Uploaded
veridelta-0.12.0.tar.gz	609.7 kB	Oct 5, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for veridelta 0.12.0
File	Interpreter	ABI	Platform	Reset
veridelta-0.12.0-py3-none-any.whl 127.4 kB Oct 5, 2026	Python 3	none	any	Details

Total release size: 737.1 kB

Release files / veridelta-0.12.0.tar.gz

Download URL	veridelta-0.12.0.tar.gz
Size	609.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`070718405ab09de02c3f54a3fdb65e839bea8d2355666e5c4293b43b5d60d97d`
BLAKE2b-256 checksum How to use checksums	`0e7bc3acd7c16411ad4cc6edb91173161c354c2316bb4496a0bf46a5b65b391b`
Upload date	Oct 5, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / veridelta-0.12.0-py3-none-any.whl

Download URL	veridelta-0.12.0-py3-none-any.whl
Size	127.4 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9afbbd1884ca94264262e23c1e29caafb61cefe71a9117e7304465dcbf2201ed`
BLAKE2b-256 checksum How to use checksums	`f620ec930528833251a457a8a78d6901dc1213282e5f13df83d5ce15d27a8e43`
Upload date	Oct 5, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`