Skip to main content

Data Governance Kit (dg_kit)

dg_kit provides programmatic access to governance metadata across logical models, physical models, and data catalogs. It is designed for CI checks, governance automation, and metadata synchronization workflows.

What it supports

  • Parse physical metadata from dbt projects.
  • Parse logical and business metadata from Oracle Data Modeler exports.
  • Validate model consistency with YAML-driven convention rules.
  • Synchronize a data catalog with a Notion data source.

Requirements

  • Python >=3.10

Installation

Install the package in editable mode:

pip install -e .

Optional extras:

pip install -e ".[dbt]"
pip install -e ".[notion]"

If you use uv:

uv sync --group dev

CLI usage

The package exposes a CLI entrypoint:

dg_kit <command> --config ./dg_kit.yml

Supported commands:

  • test validates LM/PM consistency and convention rules.
  • sync syncs the local model to the remote data catalog.
  • pull pulls the remote data catalog into a local checkpoint.

Environment variables (for sync and pull)

These commands require:

export NOTION_TOKEN=...
export DATA_CATALOG_ID=...

On PowerShell:

$env:NOTION_TOKEN="..."
$env:DATA_CATALOG_ID="..."

Example commands

dg_kit test --config ./dg_kit.yml --convention ./dg_kit.convention.yml
dg_kit sync --config ./dg_kit.yml
dg_kit pull --config ./dg_kit.yml

Configuration examples

dg_kit.yml:

name: sample_project
version: v1
logical_model:
  path: ./metadata/odm_versions
physical_model:
  path: ./warehouse/dbt_project
data_catalog:
  dc_checkpoint_path: ./.artifacts
  row_property_mapping:
    id: ID
    title: Name
    type: Type
    domain: Domain
  section_name_mapping:
    description: Description
    pk_attributes_references: PK attributes
    attributes_references: Attributes
    relations_references: Relations
    linked_documents: Linked documents
    responsible_parties: Responsible parties
    pm_mapping_references: Physical mapping
    source_systems: Source systems
    parent_entity_reference: Parent entity
    data_type: Data type
    sensitivity_type: Sensitivity type
    source_entity_reference: Source entity
    target_entity_reference: Target entity

dg_kit.convention.yml:

lm_mapping_layers:
  - core
rules:
  lm_x_pm_consistency:
    severity: error
    description: Logical and physical mappings must stay consistent.
  allowed_dependencies:
    severity: warning
    description: Restrict cross-layer dependencies.
    rules:
      core: [stage, core]
  regex_by_layer:
    severity: warning
    description: Enforce table naming conventions.
    rules:
      core_tables:
        layer: core
        regex:
          string: "^core_[a-z0-9_]+$"
technical_fields:
  core:
    - created_at
    - updated_at

Python API quick examples

Build physical model from dbt:

from pathlib import Path
from dg_kit.integrations.dbt.parser import DBTParser

pm = DBTParser(Path("path/to/dbt_project"), version="v1").parse_pm()
print(pm.version, len(pm.tables))

Parse one ODM version from a versioned project directory:

from pathlib import Path
from dg_kit.integrations.odm.parser import ODMVersionedProjectParser

parser = ODMVersionedProjectParser(Path("path/to/odm_versions"))
parser.parse_version("v1", pm)
lm = parser.get_model("v1")
print(lm.version, len(lm.entities))

Development

Run local quality checks:

./scripts/ci.sh

Run tests:

pytest

Metadata

Release files for dg-kit 0.1.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dg-kit 0.1.9
File Size Uploaded Provenance Actions
dg_kit-0.1.9.tar.gz 25.1 kB Without provenance Details for dg_kit-0.1.9.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for dg-kit 0.1.9
File Interpreter ABI Platform Actions
dg_kit-0.1.9-py3-none-any.whl Python 3 none any Details for dg_kit-0.1.9-py3-none-any.whl

Total release size: 62.1 kB

Release files / dg_kit-0.1.9.tar.gz

Download URL dg_kit-0.1.9.tar.gz
Size 25.1 kB
Tags Source
SHA-256 checksum
How to use checksums
354c2070bbfbc8f72586b5e529a45e520f4a99b21411bfd1c18de21bd3412511
BLAKE2b-256 checksum
How to use checksums
8d22b477fcc0f4fc82c32fbf031d4f28f9e3da52aad804e824b2b1b2f8449737
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.12

Release files / dg_kit-0.1.9-py3-none-any.whl

Download URL dg_kit-0.1.9-py3-none-any.whl
Size 37.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
893c40f954630002da3ff36fa1964b1d250c81dd6c100f2217eb2403fb3aa389
BLAKE2b-256 checksum
How to use checksums
e1a56e8e9e71c83141e52b8d02cb92dde1906a9f39608e4ae97772f7f567d81e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.12

Security for dg-kit 0.1.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page