turbine-data 0.7.8

A CLI tool for working with data products and contracts

Turbine

Contract-driven data quality for data products.

Turbine turns a YAML data contract into running quality checks. You declare a table's schema, ownership, freshness expectations, and validity rules in one file; Turbine validates the YAML offline, checks the live database matches it, runs every quality check against the data, scores the result, and exposes everything over a REST API and dashboard.

It uses ODCS v3.1.0, so contracts round-trip with the rest of the data ecosystem.

Example

A contract is a single YAML file:

kind: DataContract
apiVersion: v3.1.0
id: orders
domain: sales
version: "1.0.0"
status: active

slaProperties:
  - property: latency
    element: public.orders.created_at
    value: 24
    unit: hour

schema:
  - name: public.orders
    properties:
      - name: order_id
        logicalType: integer
        required: true
        primaryKey: true
      - name: amount
        logicalType: number
        required: true
      - name: status
        logicalType: string
        required: true
      - name: created_at
        logicalType: timestamp
        required: true

Run every check on it:

turbine check --datasource default orders.yml

You get a per-check verdict, a quality score per dimension (completeness, accuracy, consistency, timeliness, validity), and the failing rows persisted for follow-up.

Installation

uv add "turbine-data[duckdb]"        # local files, zero credentials
uv add "turbine-data[postgres]"      # PostgreSQL
uv add "turbine-data[snowflake]"     # Snowflake
uv add "turbine-data[all]"           # every driver + dashboard

Requires Python 3.12 or newer. The PyPI package is turbine-data; the CLI is turbine.

Quick start

# 1. Scaffold a project
uv run turbine init --database duckdb

# 2. Configure credentials
cp .env.example .env

# 3. Validate and run
uv run turbine lint  src/<project>/contracts/example.yml
uv run turbine check src/<project>/contracts/example.yml --datasource default

turbine init --demo scaffolds a fully populated example project with real data and multiple contracts.

Features

• YAML contracts in ODCS v3.1.0 — schema, ownership, SLAs, quality checks in one file
• Quite a few check types — missing, duplicate, invalid values, freshness, row count, custom SQL, Python, group, and window checks (z-score, spike, flatline)
• Schema drift detection — compare your contract to the live database before running a single check
• Dimension-aware scoring — every check is weighted by its quality dimension (completeness, accuracy, consistency, timeliness, validity)
• Row-level flagging — failing rows are persisted in a per-cell bitmap matrix; query which rows failed which checks across runs
• Management API + dashboard — turbine serve exposes runs, results, scores, and flagged rows over HTTP
• Code generation — scaffold SQLModel models and FastAPI routers from contracts
• IDE support — full LSP with VS Code and soon JetBrains extensions

Management API

turbine serve --datasource default --port 8000

Endpoints under /api/v1/manage/: /contracts, /checks/run, /runs/{id}, /runs/{id}/results, /flagged-rows/{table}. Browse /api/v1/manage/docs for the interactive OpenAPI page.

turbine dashboard --port 5173

Renders the same data as charts, run history, and a flagged-rows explorer.

CLI

turbine lint     <contract.yml>                     # validate the YAML offline
turbine validate <contract.yml> --datasource <name> # compare to live database schema
turbine check    <contract.yml> --datasource <name> # lint + validate + run every check
turbine status                                      # project health, flagged-row counts
turbine bump                                        # update contract versions
turbine new      contract|datasource|check          # scaffold a new resource
turbine generate                                    # SQLModel + FastAPI from contracts

Ecosystem

Orchestrators. Run Turbine Check Runs as native steps in your workflow tool:

• dagster-turbine — Check Definitions become Dagster AssetCheckSpecs. Partitioned assets scope each Check Run to their partition window.
• airflow-turbine — TurbineOperator with a deferred trigger; tasks wait on a Check Run without blocking a worker.
• turbine-client — sync + async Python client. Use directly when you need glue beyond the integrations above.

Editors.

• VS Code extension — diagnostics, autocomplete, quick fixes, run-from-editor. Install Turbine from the Marketplace.
• JetBrains plugin — same surface for IntelliJ, PyCharm, DataGrip.

Documentation

Full documentation lives under docs/user/docs/:

• Getting started
• Concepts — contracts, Check Definition, Check Run, Run Result, scoring
• Guides — incremental mode, row flagging, CI workflows
• CLI reference

Contributing

See docs/contributing/ for dev setup, test layout, and code conventions.