Skip to main content

trueset

CI

Write a data-quality check once, run it anywhere — pandas today, PySpark and your warehouse next. One declarative check language, one Python API, one CLI.

Data quality tools today force a trade-off: dbt tests are SQL-only, Great Expectations is powerful but heavy, and each engine (pandas / Spark / SQL) tends to want its own syntax. trueset bets on a single small Backend protocol — a check is written once and runs unchanged on any engine that implements the protocol. Adding Spark or Snowflake support means writing one class, not re-authoring a single check.

Quickstart

# From source (PyPI release coming — the name is being finalized):
git clone https://github.com/trueset-dev/trueset && cd trueset
pip install -e ".[sql]"          # [sql] adds the DuckDB backend; omit for pandas-only

trueset run --data examples/orders.csv --checks examples/checks.yml
trueset list-checks

Declarative checks (checks.yml):

suite: orders_quality
checks:
  - type: not_null
    column: order_id
  - type: unique
    column: order_id
  - type: in_range
    column: amount
    min: 0
  - type: in_set
    column: status
    values: [pending, shipped, delivered, cancelled]
  - type: matches_regex
    column: email
    pattern: '[^@\s]+@[^@\s]+\.[^@\s]+'
    severity: warn        # warns, doesn't fail the run

Or the Python API:

import pandas as pd
from trueset import Suite, validate_dataframe

df = pd.read_csv("orders.csv")
result = validate_dataframe(df, "checks.yml")

print(result.passed)                    # False
print(result.to_dict())                 # JSON-ready, ship to a warehouse/dashboard

The CLI exits non-zero when any error-severity check fails, so it drops straight into CI, dbt, or Airflow.

Use it anywhere in your pipeline (not just the warehouse)

trueset is a library first. Because checks only speak to the Backend protocol and the pandas backend takes any DataFrame, you can validate data in flight — at ingestion, before loading — and fail the pipeline the instant a batch is bad, so garbage never reaches storage (the "shift-left" pattern):

from trueset import validate_dataframe

df = extract_from_api()                        # your ingestion step
result = validate_dataframe(df, "checks.yml")  # same suite, in-memory
if not result.passed:
    raise ValueError(f"bad batch, not loading: {result.counts}")   # fail the task
load_to_warehouse(df)                          # only reached if data is clean

Drop that into an Airflow PythonOperator, a Dagster op, a Prefect task, a Lambda, or a plain script. The same checks.yml then runs post-load against the warehouse (SQLAlchemyBackend) and in CI (the Action) — write once, enforce at ingestion, in transit, and at rest. See examples/pipeline_demo.py for a runnable gate. (Batch / micro-batch oriented; native Spark validation is on the roadmap.)

Quarantine bad rows, keep the clean ones flowing

Blocking the whole batch is one option; routing is better. split() partitions a batch into clean and bad rows — with the reason each bad row failed — so you can dead-letter the bad and keep moving:

from trueset import split

result = split(df, "checks.yml")     # or a Suite / dict spec
load_to_warehouse(result.good)       # clean rows proceed
dead_letter(result.bad_annotated())  # bad rows + a `_trueset_reasons` column

Only error-severity checks divert a row by default (a warn surfaces the issue without quarantining it). From the CLI: trueset run --data batch.csv --checks c.yml --quarantine bad.csv. Under the hood, checks expose a per-row failure_spec() and every backend can return the actual failing rows (backend.failing_rows(...)) — identical across pandas, DuckDB, and SQL. trueset identifies the bad rows; you decide where they go. See examples/quarantine_demo.py.

Gate your CI in three lines

trueset ships a GitHub Action, so a data-quality check becomes a required status check on every pull request:

# .github/workflows/data-quality.yml
name: data-quality
on: [pull_request]
jobs:
  trueset:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: trueset-dev/trueset@v0
        with:
          data: data/orders.csv
          checks: quality/orders.yml

The job fails the moment an error-severity check fails. Use severity: warn on a check to surface a problem without blocking the merge.

Profiling & AI-assisted authoring

Draft a check suite instead of writing one by hand:

trueset profile  --data orders.csv                 # stats + inferred semantic types
trueset suggest  --data orders.csv                 # deterministic draft suite (no AI)
trueset suggest  --data orders.csv --ai --out checks.yml           # AI copilot
trueset suggest  --data orders.csv --describe "amount can't be negative; \
    status is one of pending/shipped/delivered/cancelled"        # English -> checks

The trust rule that makes AI safe here: the copilot only ever authors checks. Every spec it returns — from profiling or natural language — is passed through the same deterministic check registry (build_check). Anything the model hallucinates or mis-configures is discarded before it can reach your data. The AI is never in the runtime pass/fail path, so your validation stays deterministic and auditable. The model is injected as a plain callable (Completer), so it's provider-agnostic and fully testable without a key.

from trueset import profile_dataframe, suggest_from_profile
from trueset.copilot import anthropic_completer, checks_from_profile

prof = profile_dataframe(df)
draft = suggest_from_profile(prof)                      # deterministic
draft = checks_from_profile(prof, anthropic_completer())  # AI (needs ANTHROPIC_API_KEY)

Portability, proven (pandas ⇄ DuckDB)

The whole bet is that a check is written once and runs on any engine. It's not a slogan -- it's tested. The identical suite, run on pandas (in-memory) and on DuckDB (SQL pushed into the database), returns the same verdict:

check                   pandas      duckdb      match
------------------------------------------------------
row_count_parity        pass(0)     pass(0)     OK
referential_integrity   fail(1)     fail(1)     OK
value_parity            fail(3)     fail(3)     OK
------------------------------------------------------
identical verdict: True
pip install -e ".[sql]"                 # adds duckdb
python examples/portability_demo.py     # runs the comparison above

The DuckDB backend implements the same Backend protocol, but every check becomes SELECT count(*) ... WHERE ... executed inside the database -- so full tables never move. tests/test_duckdb.py asserts the two engines agree.

Run against your warehouse (Postgres, Snowflake, BigQuery, …)

The SQLAlchemyBackend generalizes the DuckDB proof to anything SQLAlchemy speaks — Postgres, MySQL/MariaDB, SQLite, Snowflake, BigQuery, Redshift. Same checks, same verdicts, pushed down as SQL. Point the CLI straight at a table:

trueset run \
  --url "postgresql+psycopg://user:pw@host:5432/analytics" \
  --table public.orders \
  --checks quality/orders.yml

Or from Python:

from sqlalchemy import create_engine
from trueset import Suite, SQLAlchemyBackend

engine = create_engine("snowflake://…")          # any SQLAlchemy URL
backend = SQLAlchemyBackend(engine, "ORDERS")
result = Suite.from_yaml("orders.yml").run(backend)

tests/test_sqlalchemy.py runs the identical example suite on pandas and on a SQL database and asserts the verdicts match — the same guarantee we hold for DuckDB, now for every warehouse. Reconciliation works across engines too: the primary can be a pandas DataFrame while the reference is a Postgres table.

Cross-system reconciliation (the wedge)

The thing no single-source tool (GE, Soda, Pandera, dbt tests) is architected to do, and whose only open-source option (data-diff) was archived in 2024: validate one dataset against another system. Because checks talk only to the Backend protocol, a reconciliation check just holds a second backend -- which can be a totally different engine.

trueset reconcile \
  --data   warehouse_orders.csv \
  --checks reconcile.yml \
  --ref    source=source_orders.csv
suite: warehouse_vs_source
checks:
  - type: row_count_parity        # counts agree within tolerance
    reference: source
    tolerance: 0.0
  - type: referential_integrity   # every key traces back to the source
    column: order_id
    reference: source
    ref_column: id
  - type: value_parity            # join on key, compare actual values
    key: order_id
    columns: [amount, status]
    reference: source
    ref_key: id

value_parity reports four failure modes at once: mismatched values on shared keys, keys only in the primary, keys only in the reference, and duplicate join keys on either side (an ambiguous join is itself a reconciliation defect). Either side can be a file, DuckDB, or any SQLAlchemy warehouse — they need not share an engine.

Governance: enforcement + evidence

trueset isn't a catalog (that's DataHub/Collibra territory). It owns the half catalogs are weak on — enforcing policy and proving compliance happened. Governance is optional metadata on a check, nothing more:

- type: matches_regex
  column: email
  pattern: '[^@\s]+@[^@\s]+\.[^@\s]+'
  owner: risk-team              # accountable party
  sensitivity: pii              # public|internal|confidential|pii|pci|phi
  regulation: [gdpr, ccpa]      # regime tags
  description: "Customer email is PII and must be well-formed"

These fields change nothing about how the check runs — they ride onto the result as machine-readable, auditable evidence. trueset report then answers policy questions directly:

trueset report --data orders.csv --checks governed.yml --by sensitivity
 governance report :: orders_quality_governed  (by sensitivity)
┏━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━━┳━━━━━━━━━━━┓
┃ sensitivity  ┃ checks ┃ pass ┃ fail ┃ error ┃ status    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━━╇━━━━━━━━━━━┩
│ pii          │ 1      │ 0    │ 1    │ 0     │ VIOLATION │
│ confidential │ 2      │ 0    │ 2    │ 0     │ VIOLATION │
└──────────────┴────────┴──────┴──────┴───────┴───────────┘
⚠ 3 failing check(s) on sensitive (pii/pci/phi/confidential) data.

Group --by owner to route failures to a team, or --by regulation for a GDPR / SOX posture. --json emits the same as auditable evidence for a compliance trail.

Classification is suggested, never imposed. trueset profile infers a sensitivity for high-precision patterns (email, phone, SSN, credit card via Luhn, IBAN), and trueset suggest pre-tags the drafted checks:

column   inferred   sensitivity
email    email      pii
ssn      ssn        pii
card     credit_card pci

Both the deterministic profiler and the AI copilot only suggest tags — like every check, they are reviewed and committed by a human, never auto-applied.

Monitoring: freshness & volume

Turn one-shot validation into continuous monitoring. Persist runs, then watch for staleness and volume drift:

trueset run --url "postgresql://…" --table orders --checks orders.yml --save "postgresql://…/trueset_history"
trueset history --store "postgresql://…/trueset_history"      # the audit trail
trueset monitor --store "postgresql://…/trueset_history" --suite orders_quality --sigma 3
  • freshness is an ordinary check — the newest timestamp must be recent:
    - type: freshness
      column: created_at
      max_age_hours: 6        # fail if the table hasn't updated in 6 hours
    
  • trueset monitor trends any metric against the baseline of past runs and exits non-zero on a >sigma anomaly — a silent 90% drop in rows is caught even when every row that is there passes every check:
    trueset monitor --store "$H" --suite orders --metric rows --method mad
    trueset monitor --store "$H" --suite orders --metric failing_rows --check not_null --column email
    
    --metric can be rows, failing_rows, or total_rows (with --check/ --column to pick a check); --method is zscore (mean/std) or mad (median-absolute-deviation — robust when a few past runs were themselves outliers). Detection is deterministic and explainable — never a black box.

Works with your stack (dbt today)

trueset composes with the tools you already run — it doesn't replace them. If you have dbt tests, adopt trueset without rewriting a single one:

trueset import-dbt --schema models/schema.yml --model orders --out orders.yml
trueset run --url "$WAREHOUSE" --table analytics.orders --checks orders.yml

not_null/unique map directly, accepted_values→in_set, relationships→referential_integrity, and dbt test severity is preserved. Custom/singular dbt tests are reported (never silently dropped), and every imported check is validated so the output always runs. Point trueset at the tables dbt builds and you also get cross-engine portability, reconciliation against the source, governance, and monitoring — on top of your existing tests.

Built-in checks

columns_exist, not_null, unique, in_set, in_range, matches_regex, row_count, no_duplicate_rows, freshness, metric, plus the reconciliation checks row_count_parity, referential_integrity, value_parity. Each has a severity of error (default) or warn.

Architecture

YAML / Python API
        │
     Suite ── list of ── Check          checks speak ONLY to the Backend
        │                  │            protocol, never to an engine
        ▼                  ▼
   SuiteResult  ◀──  Backend protocol
                          ▲
        ┌────────────┬───────┴───────┬────────────┐
   PandasBackend  DuckDBBackend  SQLAlchemyBackend  (SparkBackend)
     [built]        [built]     [built: any warehouse]  [next]

The Backend protocol (src/trueset/backends/base.py) is deliberately tiny: row_count, null_count, distinct_count, count_out_of_range, etc. A SQL backend implements each as a pushed-down SELECT count(*) ... WHERE ... so data never leaves the warehouse.

Roadmap

  • Backends: pandas, DuckDB, and SQLAlchemy (any warehouse).
  • Governance: owner/sensitivity/regulation metadata + policy reports.
  • Results sink: write run history to a table for trend/anomaly monitoring.
  • Freshness & volume checks for pipeline monitoring.
  • Interop: import dbt tests + Great Expectations / Soda suites.
  • Backends: PySpark.

Development

pip install -e ".[dev]"
pytest

License

Apache-2.0 (recommended for broad corporate adoption).

Metadata

Release files for trueset 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for trueset 0.1.0
File Size Uploaded Provenance Actions
trueset-0.1.0.tar.gz 68.2 kB With provenance Details for trueset-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for trueset 0.1.0
File Interpreter ABI Platform Actions
trueset-0.1.0-py3-none-any.whl Python 3 none any Details for trueset-0.1.0-py3-none-any.whl

Total release size: 121.4 kB

Release files / trueset-0.1.0.tar.gz

Download URL trueset-0.1.0.tar.gz
Size 68.2 kB
Tags Source
SHA-256 checksum
How to use checksums
76751195069741d2acb91bb0619160a3d2cbd721fedf130f4c04c93315a961c6
BLAKE2b-256 checksum
How to use checksums
f367fd5137703e57c8068152ae8a8c0cb9917e67f101d47dcad2e6935819d02b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 25, 2026.

Transparency log

Release files / trueset-0.1.0-py3-none-any.whl

Download URL trueset-0.1.0-py3-none-any.whl
Size 53.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
9222772ab3260fda04945d59345b8fa6ed8f13c96a4f8d5b8c1616f87755a41b
BLAKE2b-256 checksum
How to use checksums
3551c215c4d675a499206141fc435f84a5401ef4c4da1b64607aa04673f484d0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 25, 2026.

Transparency log

Security for trueset 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for trueset-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 25, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page