clinical-contract 0.1.5

Valide des data contracts YAML cliniques contre des fichiers Parquet

clinical-contract

Ensure your data matches the expectations defined in YAML contracts — check schemas, data types, and quality rules automatically on Parquet and CSV files.

---

Overview

clinical-contract is a data contract validation library designed for clinical and healthcare data pipelines. It bridges the gap between data documentation and data quality enforcement by allowing teams to define their data expectations in a human-readable YAML contract and automatically verify those expectations against real Parquet and CSV files.

A contract defines:

• Schema — which columns exist, their logical and physical types
• Quality rules — SQL-based assertions that must hold true on the data

The library is DuckDB-first and is compatible with PyScript, making it suitable for both server-side pipelines and browser-based tooling.

---

Features

• YAML contract validation — verify that a contract file is structurally complete before running it against data
• Schema verification — check that required columns exist in the Parquet or CSV file with compatible types
• SQL quality checks — execute custom SQL assertions and report pass/fail with obtained vs expected values
• Flexible type mapping — loose type family matching (string, varchar, text are treated as equivalent; int32, int64, integer likewise)
• DuckDB engine — one execution path for schema checks and SQL quality checks
• PyScript compatible — runs in the browser via Pyodide/WebAssembly
• Clean CLI output — formatted tables with ✅/❌ indicators directly in the terminal
• Programmable API — use as a Python library in your own pipelines and CI workflows

---

Installation

pip install clinical-contract

duckdb is installed as a core dependency.

---

Quick Start

1. Write a contract

# datacontract.yaml
apiVersion: v3.1.0
kind: DataContract
id: export-contract
name: Export Contract
version: 1.0.0
status: active
description:
  purpose: "Export dataset containing medical events and sampling data"
  usage: "Analytics and downstream processing"
  limitations: "Historical data may contain legacy timestamps"

schema:
  - name: S
    physicalType: TABLE
    description: Exported dataset containing patient event data
    properties:
      - name: IPP
        logicalType: string
        physicalType: TEXT
        description: Permanent patient identifier
        required: true
        quality:
          - type: sql
            description: IPP must not be null
            query: "SELECT COUNT(*) FROM export WHERE IPP IS NULL"
            mustBe: 0
          - type: sql
            description: IPP length must be between 35 and 37 characters
            query: "SELECT COUNT(*) FROM export WHERE LENGTH(IPP) NOT BETWEEN 35 AND 37"
            mustBe: 0

      - name: EVENT_DATE
        logicalType: date
        physicalType: DATE
        description: Medical event date
        required: true
        quality:
          - type: sql
            description: No dates in the future
            query: "SELECT COUNT(*) FROM export WHERE EVENT_DATE > CURRENT_DATE"
            mustBe: 0

2. Validate the contract structure

clinical-contract validate datacontract.yaml

📋  Validation de la structure : datacontract.yaml

┌─────────────┬────────┬──────────────────────┐
│ Champ       │ Statut │ Valeur               │
├─────────────┼────────┼──────────────────────┤
│ apiVersion  │ ✅     │ v1.0.0              │
│ kind        │ ✅     │ DataContract        │
│ id          │ ✅     │ export-contract     │
│ name        │ ✅     │ Export Contract     │
│ version     │ ✅     │ 1.0.0               │
│ status      │ ✅     │ active              │
│ description │ ✅     │ présent             │
│ schema      │ ✅     │ 6 colonnes détectées│
└─────────────┴────────┴──────────────────────┘

✅  Structure valide — tous les champs sont présents.

3. Run checks against a Parquet file

clinical-contract check datacontract.yaml export.parquet

🔍  Vérification du contrat
    Contrat : datacontract.yaml
    Parquet : export.parquet

── Vérification du schéma ──────────────────────────────────────

  Schema : export
  ┌─────────────┬───────────┬──────────────┬──────────┐
  │ Colonne     │ Type YAML │ Type Parquet │ Statut   │
  ├─────────────┼───────────┼──────────────┼──────────┤
  │ IPP         │ string    │ string       │ ✅       │
  │ EVENT_DATE  │ date      │ date32       │ ✅       │
  └─────────────┴───────────┴──────────────┴──────────┘

  ✅ 2/2 colonnes valides

── Quality checks ──────────────────────────────────────────────

  ┌────────┬────────────┬──────────────────────────────┬──────────┬────────┬─────────┐
  │ Schema │ Property   │ Description                  │ Résultat │ Obtenu │ Attendu │
  ├────────┼────────────┼──────────────────────────────┼──────────┼────────┼─────────┤
  │ export │ IPP        │ IPP must not be null         │ ✅       │ 0      │ 0       │
  │ export │ IPP        │ IPP length 35-37 characters  │ ❌       │ 3      │ 0       │
  │ export │ EVENT_DATE │ No dates in the future       │ ✅       │ 0      │ 0       │
  └────────┴────────────┴──────────────────────────────┴──────────┴────────┴─────────┘

  2/3 checks passés.

---

CLI Reference

clinical-contract validate <contract.yaml>

The validate command verifies that a YAML contract file is correctly written and conforms to the Open Data Contract Standard v3.1.0. This ensures that all required fields are present and correctly structured.

Required top-level fields: apiVersion, kind, id, name, version, status, description, schema

Expected sub-fields:

• description must include: purpose, usage, limitations
• schema must include for each item: name, physicalType, description, properties
• properties (inside each schema) must include: name, logicalType, physicalType, description

Exit codes: 0 if valid, 1 if any required field is missing or invalid (including unsupported logicalType).

---

clinical-contract check <contract.yaml> <data_file> [backend]

Runs a full validation pipeline in three stages against a Parquet or CSV file:

1. YAML structure — same checks as validate
2. Schema compatibility — verifies that required columns exist in the Parquet or CSV file with compatible types. Quality checks are blocked if this step fails.
3. Quality checks — executes each SQL assertion and reports the result

Backend options: auto (default), duckdb

Exit codes: 0 if all checks pass, 1 if any check fails or a column is missing/mistyped, 2 if an execution error occurs.

---

Type Mapping

Types in the YAML contract are matched with a hybrid strategy:

• Strict integer-width matching for explicit integer types: int8, int16, int32, int64, uint8, uint16, uint32, uint64
• Family-based matching for generic types like integer, string, timestamp, etc.

uint32 is accepted in YAML and normalized to DuckDB canonical uinteger before comparison.

YAML logical type	Compatible Parquet types
string, text, varchar	string, large_string, utf8, large_utf8
integer, int	int8, int16, int32, int64, uint8, uint16, uint32, uint64, tinyint, smallint, integer, bigint, utinyint, usmallint, uinteger, ubigint
int8, int16, int32, int64, uint8, uint16, uint32, uint64	Strict canonical match (int32 ↔ integer, uint32 ↔ uinteger, etc.)
float, double, decimal	float32, float64, double, decimal128
boolean, bool	bool, boolean
date, date32	date32, date64
datetime, timestamp	timestamp[ms], timestamp[us], timestamp[ns], timestamp[s], timezone variants
binary, bytes	binary, large_binary

---

Python API

Beyond the CLI, clinical-contract can be used directly in Python pipelines:

from clinical_contract import load_contract

# Load and parse the contract
contract, raw = load_contract("datacontract.yaml")

# Validate structure only
from clinical_contract import DataContract
validate_report = DataContract.validate_structure(raw)
if not validate_report.success:
    for f in validate_report.missing():
        print(f"Missing field: {f.field}")

# Check schema compatibility
schema_reports = contract.check_schema("export.parquet")
for report in schema_reports:
    if not report.success:
        for col in report.failures():
            print(f"{col.column}: {col.status_icon}")

# Run quality checks
report = contract.check("export.parquet", backend="duckdb")

print(f"Success: {report.success}")
print(f"Code: {report.code}")  # 0 = pass, 1 = fail, 2 = error

for result in report.failed():
    print(f"  ❌ {result.description}")
    print(f"     obtained={result.obtained}, expected={result.expected}")

---

Contract Schema Reference

apiVersion: string        # Contract specification version (e.g. v3.1.0)
kind: DataContract        # Must be "DataContract"
id: string                # Unique identifier for this contract
name: string              # Human-readable name
version: string           # Data version (semver recommended)
status: string            # active | draft | deprecated

description:
  purpose: string         # Why this dataset exists
  usage: string           # How it should be used
  limitations: string     # Known limitations or caveats

schema:
  - name: string          # Table/view name (used in SQL queries)
    physicalType: TABLE   # TABLE | VIEW
    description: string
    properties:
      - name: string          # Column name (case-sensitive)
        logicalType: string   # Semantic type (string, integer, date…)
        physicalType: string  # Storage type (TEXT, INT, DATE…)
        description: string
        required: bool        # Default: false (missing optional columns do not fail schema check)
        quality:              # Optional list of SQL assertions
          - type: sql
            description: string   # Human-readable description of the rule
            query: string         # SQL returning a single COUNT(*)
            mustBe: integer       # Expected result (usually 0)

---

Development

# Clone the repository
git clone https://github.com/artheioupfat/clinical-contract.git
cd clinical-contract

# Create a virtual environment
uv venv
source .venv/bin/activate

# Install in editable mode with all dependencies
uv run pip install -e .

#Installer les dépendances dev
uv sync --extra dev

#lancer les tests 
pytest -v

---

License

MIT — see LICENSE for details.

---

Author

Arthur PRIGENT — GitHub