Data Engineering Toolkit compiler

The det CLI turns the controlled V4.0.3 official-ODCS Excel superset into canonical ODCS, DET execution metadata, and a readable dbt project. It includes safe questionnaire-based workbook creation, ODCS/DDL/dbt-to-Excel import and merge-safe synchronization, source-schema import, cross-sheet semantic/type validation, atomic post-sync generation, Data Contract synchronization, SQLFluff, and the isolated det prove acceptance flow. Data Contract CLI, dbt Core, DuckDB, and SQLFluff are default dependencies.

V4.0.3 makes the visible official Quality sheet authoritative, preserves advanced per-rule ODCS values, supports eight production warehouse providers, and adds mandatory 85.01% Python statement coverage. The generated dbt package has 98.53% DuckDB-scoped macro implementation coverage plus 100% public-API direct execution coverage and supports dbt Core plus Fusion-compatible projects. External resources are isolated behind brokers; reusable services interpret workbooks and input formats; independent components validate with stable diagnostic codes; and one emitter owns each artifact family. The CLI remains a thin exposer over ToolkitApplication.

Install

python -m pip install dbt-data-engineering-toolkit-compiler==4.0.3

The default installation includes Data Contract CLI, dbt Core, dbt-duckdb, DuckDB, and SQLFluff. Install one additional warehouse adapter per deployment environment, for example:

python -m pip install "dbt-data-engineering-toolkit-compiler[snowflake]"

Supported extras are athena, bigquery, clickhouse, databricks, duckdb, postgres, redshift, snowflake, and spark.

Workflow

Use this sequence to move from workbook design to validated, testable dbt delivery.

Step	Business action	Outcome	Command
1	Run the guided, safe workbook builder	Product basics; no demo rows by default	`det workbook build PRODUCT.xlsx`
2	Import or define target structures	ODCS model and property rows	`det workbook import ...`
3	Register/import upstream sources	Checked source fields and types	`det source import ...`
4	Define model inputs, mappings, and joins	Context-aware fields and executable cardinality checks	`det workbook refresh PRODUCT.xlsx`
5	Add contract rules on Quality; runtime handling on Operational Validation	ODCS tests, warnings, and quarantine with one clear authority	Use workbook sheets (no CLI command)
6	Validate the workbook	Actionable cross-sheet diagnostics	`det validate PRODUCT.xlsx`
7	Preview generated changes	No files changed	`det generate PRODUCT.xlsx --project-dir dbt_product --dry-run --prune`
8	Export package environment variables, then generate, sync, and lint	dbt packages resolve from your published Git URL and pinned revision	`export DBT_DATA_ENGINEERING_TOOLKIT_GIT_URL=https://github.com/systemizing-solutions/dbt_data_engineering_toolkit.git; det generate ...; dbt deps; det check ...`
9	Run isolated acceptance	Contract, dbt tests, evaluator	`det prove PRODUCT.xlsx --project-dir dbt_product`
10	Merge later schema changes	Mappings and rules preserved	`det workbook sync ...`
Demo	Opt in only when explicitly wanted	Customer 360 training rows	`det workbook build demo.xlsx --sample-customer-data`

Before running dbt deps, set the package source URL used by generated packages.yml:

export DBT_DATA_ENGINEERING_TOOLKIT_GIT_URL=https://github.com/systemizing-solutions/dbt_data_engineering_toolkit.git

If you need a custom variable name, set Toolkit Git Env on the DET Build sheet before det generate, then export that variable name in your shell.

Quick start

det workbook build data_product.xlsx --no-input
det validate data_product.xlsx

The default is blank and does not add Customer 360 sample data. Use --sample-customer-data only when you explicitly want a demonstration workbook.

Continue with det --help and det workbook --help. The source repository root README contains the complete workbook-to-ODCS-to-dbt walkthrough, architecture, compatibility matrix, testing definitions, and deployment guide.

Development Status
- 5 - Production/Stable
Environment
- Console
Programming Language
Topic
- Database
Typing
- Typed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz (268.6 kB view details)

Uploaded Sep 9, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

dbt_data_engineering_toolkit_compiler-4.0.3-py3-none-any.whl (290.3 kB view details)

Uploaded Sep 9, 2026 Python 3

File details

Details for the file dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz.

File metadata

Download URL: dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz
Upload date: Sep 9, 2026
Size: 268.6 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz
Algorithm	Hash digest
SHA256	`5aeef41fcc2869555d4b9f04deb0332ff0c74c1cb3b6c11c1aec9f6c09052843`
MD5	`dd4ad4fc576c69aaa604a76b8e0c58d7`
BLAKE2b-256	`69aea65bb76e1d2811fba65abe3ae8f2d9bc1ff24435a74bdf04edebcebcd4ca`

See more details on using hashes here.

Provenance

The following attestation bundles were made for dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz:

Publisher: release.yml on systemizing-solutions/dbt_data_engineering_toolkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: dbt_data_engineering_toolkit_compiler-4.0.3.tar.gz
- Subject digest: 5aeef41fcc2869555d4b9f04deb0332ff0c74c1cb3b6c11c1aec9f6c09052843
- Sigstore transparency entry: 2772165570
- Sigstore integration time: Sep 9, 2026
Source repository:
- Permalink: systemizing-solutions/dbt_data_engineering_toolkit@19520fd959dba1267d18e5a66cc56e82b029234c
- Branch / Tag: refs/heads/main
- Owner: https://github.com/systemizing-solutions
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@19520fd959dba1267d18e5a66cc56e82b029234c
- Trigger Event: workflow_run