Veridelta compares two datasets on their primary keys and reports every row that differs under the rules you declare. Nothing is forgiven unless a rule says so, and the exit code tells CI whether the datasets match. Use it to verify a migration, a pipeline change, or a model's new evaluation run, on a laptop, in CI, or inside a warehouse.
Watch the 60-second demo: a port from IDL to Python must match one to one, every row differs, and only the real bug fails CI.
It runs on Polars. Read the documentation.
Features
- Declared rules. Tolerances, null sentinels, regular expressions, value maps, date parsing, casts, and fuzzy text matching apply in nine fixed stages. Nothing is forgiven unless a rule says so, and
strict_typesfails a column whose type drifts. What rules cannot do lists the checks they leave out, such as arithmetic and comparing two columns. - Comparison inside the warehouse. Two tables in one Snowflake, Databricks, or BigQuery warehouse are compared where they are stored, as are two Postgres, SQL Server, Redshift, or DuckDB tables that set
pushdown. The rules compile to SQL, and only counts and keys come back. Pushdown lists the exceptions and the services the SQL has run in. - Many sources. CSV, Parquet, JSON, NDJSON, Arrow, Avro, and Excel files, Delta Lake and Iceberg tables, DuckDB files and MotherDuck databases, and Postgres, MySQL, SQL Server, Oracle, SQLite, and other databases. Files and tables are scanned lazily where Polars can.
- Built for CI. Exit codes, a JSON summary, a standalone HTML report, a Markdown summary for pull requests, OpenTelemetry metrics, and files of the rows that differ. A GitHub Action posts the summary on each pull request, and a GitLab CI template on each merge request when it has a token.
- Checks before a run.
veridelta validatereports what would stop a run without reading any rows, and a JSON Schema gives editors completion for configuration files. - For AI agents. An agent runs the same checks and comparisons through the command line, or through
veridelta mcp, a Model Context Protocol server that reads files only from the folders you name. AI agents gives the steps.
Status
Veridelta is in alpha, so a minor version can still change the configuration or the Python API. Its tests run on generated data, including a suite that seeds known drift into two datasets and checks that each run reports exactly that drift, in a local run and in pushdown. Nobody but its maintainer is known to have run it yet. Its pushdown SQL has run in DuckDB, Postgres 16, and SQL Server 2022, but not yet in a live cloud warehouse, as Pushdown says.
Install
uv add veridelta # or: pip install veridelta
uv add 'veridelta[snowflake]' # extras: snowflake, databricks, bigquery, delta, iceberg, database, duckdb, excel, fuzzy, mcp, all
Quick start
The smallest configuration names the two files and the keys that pair their rows. The suffix of each path says what format it is:
# veridelta.yaml
primary_keys: [id]
source:
path: legacy.csv
target:
path: modern.csv
The recording above runs this file on two three-row files. validate checks the file without reading any rows, and run compares the two files. It exits 0 when they match, 1 when rows differ, and 3 when the run could not finish:
veridelta validate -c veridelta.yaml
veridelta run -c veridelta.yaml
Two files that need no rules need no configuration file either. Name them and the key that pairs their rows, and each format still follows its suffix:
veridelta run legacy.csv modern.csv --key id
Rules say what counts as a match, column by column. This file forgives one percent on a total and compares phone numbers on their digits alone:
primary_keys: ["transaction_id"]
source:
path: "legacy.parquet"
target:
path: "modern.parquet"
rules:
- column_names: ["grand_total"]
relative_tolerance: 0.01
- column_names: ["contact_number"]
regex_replace: {"[^0-9]": ""}
In Python, DiffEngine compares two LazyFrames with the same settings a file holds:
import polars as pl
from veridelta import DiffConfig, DiffEngine, DiffRule
result = DiffEngine(
DiffConfig(
primary_keys=["user_id"],
rules=[DiffRule(pattern="^AMT_.*", absolute_tolerance=0.05)],
),
pl.scan_parquet("legacy.parquet"),
pl.scan_parquet("modern.parquet"),
).run()
if not result.summary.is_match:
raise SystemExit(f"{result.summary.changed_count} rows differ")
Documentation
- Tutorials: six notebooks, from a first comparison in Python to a CI pipeline.
- How-to guides: one task each, such as turning a failing first run into rules.
- User guide: the command line, configuration, rules, results, sources, pushdown, and AI agents.
- CI integrations: the GitHub Action and the GitLab CI template.
- API reference: the public Python interface.
- Upgrading: what to change when a release breaks something.
- Roadmap: work that is not built yet.
Accessibility
ACCESSIBILITY.md states what Veridelta aims for, the barriers known today, and how to report one.
Contributing
See CONTRIBUTING.md for the development setup and the checks a change must pass.
License
Apache 2.0.
1 maintainer
Metadata
1 maintainer
Release files for veridelta 0.41.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| veridelta-0.41.0.tar.gz | 1.1 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| veridelta-0.41.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.3 MB
Release files / veridelta-0.41.0.tar.gz
| Download URL | veridelta-0.41.0.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a65f14e7f504896a5b1c82620c2bf77119e334f88bceb0246e89793c3fd3b848
|
|
BLAKE2b-256 checksum How to use checksums |
f2a8dc6a8541d6e028baaae9d1654b25d86783023e826fddb6c93aea7a170fc1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.
Transparency logRelease files / veridelta-0.41.0-py3-none-any.whl
| Download URL | veridelta-0.41.0-py3-none-any.whl |
|---|---|
| Size | 178.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
80554479b337a368631c778f36e0b74a31902f8f4b33a8001c77e85360cd7ea0
|
|
BLAKE2b-256 checksum How to use checksums |
f0134a0ec4ffbb2e9fa8430ccdc0306896954a3fec4c6abb019529ed5efd7294
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.
Transparency log
