Verifily

ML data quality gate. Ingest, validate, and ship datasets with confidence.

Verifily catches contamination, PII leaks, SQL template leakage, contract violations, and metric regressions before they reach production. It runs locally — no network, no GPU, no external services.

One command gates your CI pipeline. Exit 0 means ship.

Install

pip install -e .

For integrations (HuggingFace, W&B, MLflow) and API server:

pip install -e ".[all]"

60-Second Quick Start

# 1. Scaffold a project
verifily quickstart my_project

# 2. Ingest raw data (JSONL, CSV, Parquet, or HuggingFace)
verifily ingest --in my_project/data/raw/sample.csv \
                --out my_project/data/artifact \
                --schema sft

# 3. Run the CI gate
verifily pipeline --config my_project/verifily.yaml --ci
# Exit 0 = SHIP, 1 = DONT_SHIP, 2 = INVESTIGATE

Or run the full demo end-to-end:

bash scripts/demo_quickstart_ci.sh

What Verifily Prevents

Supported Schemas

8 canonical dataset types, auto-detected from field names:

CLI Commands

NL2SQL Commands

Integrations

All opt-in with lazy imports. No hard dependencies.

# HuggingFace
verifily ingest --in "hf://squad" --out datasets/squad --schema qa

# W&B + MLflow
verifily pipeline --config pipeline.yaml --wandb --mlflow

CI Exit Codes

Documentation

• Product Overview
• Quick Install
• 3-Minute Quickstart
• Decision Gate
• Dataset Fingerprints
• CI Init
• API & Jobs
• Monitor
• Versioning & Stability
• Changelog

Versioning

Verifily follows Semantic Versioning. See VERSIONING.md.

Current version: 1.4.1

Stability Guarantees

• Deterministic outputs — fixed seed produces identical results across runs
• Stable contracts — run_contract_v1 schema is frozen within the v1.x line
• Stable exit codes — 0/1/2/3/4 semantics are frozen
• Backward compatibility within MAJOR version — artifacts from any v1.x release are accepted
• 1,300+ tests — all deterministic, no network, no GPU

License

Business Source License 1.1 (BSL-1.1). See LICENSE for details.

You may use Verifily for any purpose except offering it as a commercial data quality or ML pipeline gating service to third parties. On 2030-02-16, the license converts to Apache 2.0.