turbine-data 0.4.0

A CLI tool for working with data products and contracts

Turbine

Contract-driven data quality for data products — powered by ODCS and Soda Core.

How It Works

Contract ➜ Lint ➜ Check ➜ Score ➜ Flag ➜ Observe

1. Contract — Define expectations in YAML using ODCS v3.1.0
2. Lint — Validate contract schema before anything touches a database
3. Check — Run quality checks against live data (SodaCL, SQL, Python, window, group)
4. Score — Calculate a dimension-aware quality score
5. Flag — Tag failing rows with bitmask flags for downstream filtering
6. Observe — Export traces and metrics via OpenTelemetry

Features

• YAML contracts — ODCS v3.1.0 with Soda extensions for quality checks
• 13 check types — missing, duplicate, invalid, freshness, row_count, SQL, Python, typed multi-table Python, group, and window (zscore, spike, flatline)
• Schema drift detection — Compare live database schemas against your contract
• Dimension-aware scoring — Weight quality dimensions (completeness, accuracy, …) per check
• Row-level flagging — Per-cell Roaring-bitmap matrix tracks which rows failed which checks across runs
• Code generation — Scaffold SQLModel models and FastAPI routers from contracts
• Dependency management — Lockfile-based contract dependency resolution
• IDE support — Language server (LSP) with extensions for VSCode and JetBrains

Quick Start

Prerequisites: Python 3.13+ and uv

# Install with your database driver
uv add "turbine-data[snowflake]"    # or: postgres, duckdb

# Initialize the recommended src/{project_name} layout
uv run turbine init --defaults

# Copy .env.example to .env and fill your warehouse credentials
cp .env.example .env

# Validate the starter Contract
uv run turbine lint src/{project_name}/contracts/example.yml

# Run quality checks against the starter Datasource named default
uv run turbine check --datasource default src/{project_name}/contracts/example.yml

Supported Databases

Database	Install extra
PostgreSQL	turbine-data[postgres]
Snowflake	turbine-data[snowflake]
DuckDB	turbine-data[duckdb]

Documentation

Full docs live in docs/user/docs/ — covering getting started, guides, concepts, and CLI reference.

Contributing

See the contributing guide for dev setup, testing, and code style.