Veridelta

Veridelta compares two datasets on their primary keys and reports every row that differs once the rules you declare are applied. Use it to verify a system migration, a model retrain, or a pipeline change.

It runs on Polars. Read the documentation.

Features

Declared rules. Tolerances, null sentinels, regular expressions, value maps, date parsing, casts, and fuzzy text matching apply in nine fixed stages. Nothing is forgiven unless a rule says so, and strict_types fails a column whose type drifts.
The same verdict in the warehouse. Two tables in Snowflake, Databricks, BigQuery, Postgres, or DuckDB are compared where they are stored. The rules compile to SQL, and only counts and keys come back.
Many sources. CSV, Parquet, JSON, Arrow, Avro, and Excel files, Delta Lake and Iceberg tables, DuckDB files and MotherDuck databases, and Postgres, MySQL, SQL Server, Oracle, SQLite, and other databases. Files and tables are scanned lazily where Polars can.
Built for CI. Exit codes, a JSON summary, a standalone HTML report, a Markdown summary for pull requests, OpenTelemetry metrics, and files of the rows that differ. A GitHub Action and a GitLab CI template post the summary on each pull request.
Checks before a run. veridelta validate reports what would stop a run without reading any rows, and a JSON Schema gives editors completion for configuration files.

Install

uv add veridelta                # or: pip install veridelta
uv add 'veridelta[snowflake]'   # extras: snowflake, databricks, bigquery, delta, iceberg, database, duckdb, excel, fuzzy, all

Quick start

In Python, DiffEngine compares two LazyFrames:

import polars as pl
from veridelta import DiffConfig, DiffEngine, DiffRule

result = DiffEngine(
    DiffConfig(
        primary_keys=["user_id"],
        rules=[DiffRule(pattern="^AMT_.*", absolute_tolerance=0.05)],
    ),
    pl.scan_parquet("legacy.parquet"),
    pl.scan_parquet("modern.parquet"),
).run()

if not result.summary.is_match:
    raise SystemExit(f"{result.summary.changed_count} rows differ")

The same comparison as a YAML file, for the CLI and CI:

# veridelta.yaml
primary_keys: ["transaction_id"]
source:
  path: "legacy.parquet"
  format: "parquet"
target:
  path: "modern.parquet"
  format: "parquet"
rules:
  - column_names: ["grand_total"]
    relative_tolerance: 0.01
  - column_names: ["contact_number"]
    regex_replace: {"[^0-9]": ""}

validate checks the file without reading any rows, and run compares the datasets:

veridelta validate -c veridelta.yaml
veridelta run -c veridelta.yaml

Documentation

Tutorials: five notebooks, from a first comparison in Python to a CI pipeline.
User guide: configuration, sources, rules, pushdown, results, and the command line.
CI integrations: the GitHub Action and the GitLab CI template.
API reference: the public Python interface.
Roadmap: work that is not built yet.

Contributing

See CONTRIBUTING.md for the development setup and the checks a change must pass.

License

Apache 2.0.

Metadata

Release files for veridelta 0.13.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for veridelta 0.13.0
File	Size	Uploaded
veridelta-0.13.0.tar.gz	642.2 kB	Oct 5, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for veridelta 0.13.0
File	Interpreter	ABI	Platform	Reset
veridelta-0.13.0-py3-none-any.whl 114.1 kB Oct 5, 2026	Python 3	none	any	Details

Total release size: 756.4 kB

Release files / veridelta-0.13.0.tar.gz

Download URL	veridelta-0.13.0.tar.gz
Size	642.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`5f2346bd0d706fc36f6f5b3dfac6a3366a861f8205b9a6d291fad401707b6cd5`
BLAKE2b-256 checksum How to use checksums	`cb6d11f774bacb8f50b7881838d4a195af9db8a29ef77cd0cf91e9e221f924aa`
Upload date	Oct 5, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / veridelta-0.13.0-py3-none-any.whl

Download URL	veridelta-0.13.0-py3-none-any.whl
Size	114.1 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`bcf6de124d6f3c645e55f2ee81c464f8f6eb133a5de51444e4a9972ff83deb36`
BLAKE2b-256 checksum How to use checksums	`d51261fb5a79b9776d5f4fb382e3018fc11e755dcb5f500df53b4c68b14b3a4f`
Upload date	Oct 5, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`