dagster-rocky

Dagster integration for Rocky, the typed graph between your code and your warehouse.

dagster-rocky wraps the rocky CLI as a Dagster ConfigurableResource and exposes Rocky-managed tables as materializable Dagster assets. Compile-time contracts, column-level lineage, schema-drift detection, quality check results, and per-model cost surface as native Dagster events, so the guarantees Rocky enforces at compile time appear directly in the Dagster asset graph.

Two behaviors worth knowing. Rocky's per-check severity maps to Dagster's, so a failing advisory check emits WARN instead of paging anyone. And when the engine's failure containment is enabled, the models Rocky withholds behind a failed upstream appear as AssetObservation events naming the blocking model, so a partial run reads honestly in the asset graph.

Install

uv add dagster-rocky

You'll also need the Rocky CLI on your $PATH: https://github.com/rocky-data/rocky/releases?q=engine

Quick start (component)

Add a defs.yaml next to your other Dagster definitions:

type: dagster_rocky.RockyComponent
attributes:
  binary_path: rocky
  config_path: config/rocky.toml
  models_dir: models

Dagster's component loader will:

  1. Run rocky discover (and rocky compile, when models are present) and cache the result.
  2. Build one subset-aware multi_asset per Rocky group, with declared row_count / column_match / freshness checks per table.
  3. On materialization, shell out to rocky run --filter <key>=<value> for the selected subset and yield MaterializeResult + AssetCheckResult events with rich metadata.

Quick start (resource)

import dagster as dg
from dagster_rocky import RockyResource

rocky = RockyResource(
    binary_path="rocky",
    config_path="config/rocky.toml",
    timeout_seconds=3600,
)

defs = dg.Definitions(resources={"rocky": rocky})

Then in an asset:

@dg.asset
def acme_orders(rocky: RockyResource) -> dg.MaterializeResult:
    result = rocky.run(filter="tenant=acme")
    return dg.MaterializeResult(
        metadata={"tables_copied": result.tables_copied, "duration_ms": result.duration_ms},
    )

Public API

Symbol Purpose
RockyResource ConfigurableResource wrapping the Rocky CLI
RockyComponent State-backed Dagster component that loads Rocky tables as assets
RockyDagsterTranslator Subclass to customize asset key / group / tag mapping
RockyMetadataSet Namespaced metadata (source_id, strategy, watermark, …)
load_rocky_assets() Functional helper that returns AssetSpec for each Rocky table
emit_materializations() / emit_check_results() Convert a RunResult into Dagster events
check_metadata() Build a metadata mapping for a single Rocky CheckResult
cost_metadata_from_optimize() Extract per-model cost recommendations from OptimizeResult
parse_rocky_output() Auto-detect a Rocky JSON payload and return the matching Pydantic model

Documentation

Related projects

License

Apache 2.0

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.61.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for rocky-data from gravatar.com rocky-data

Credits

Author: Hugo Correia

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.11

Tags

dagster data-engineering dbt rocky sql transformation

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dagster_rocky-1.61.0.tar.gz (368.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dagster_rocky-1.61.0-py3-none-any.whl (126.7 kB view details)

Uploaded Python 3

File details

Details for the file dagster_rocky-1.61.0.tar.gz.

File metadata

  • Download URL: dagster_rocky-1.61.0.tar.gz
  • Upload date:
  • Size: 368.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dagster_rocky-1.61.0.tar.gz
Algorithm Hash digest
SHA256 63cdd8ec50144b090c73cdaf179cd981c25941a36af4f0d02a15149c5736b215
MD5 144c40f99d946a0688ce54f3314bb4bc
BLAKE2b-256 410ab693bf02b8cd396fe9d9cff81433f6716287831db430ba402c1488ad712e

See more details on using hashes here.

Provenance

The following attestation bundles were made for dagster_rocky-1.61.0.tar.gz:

Publisher: dagster-release.yml on rocky-data/rocky

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dagster_rocky-1.61.0-py3-none-any.whl.

File metadata

  • Download URL: dagster_rocky-1.61.0-py3-none-any.whl
  • Upload date:
  • Size: 126.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dagster_rocky-1.61.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b82c72e25a037f40db2f6fd596ea8e86eff1a33d16fd77655c30027ce33b1b24
MD5 73a12d8d5225527a355d422d739b4f76
BLAKE2b-256 b77ab86bb01f39d93f6060b3238d47d3936904e560d4dd0ad7ccb1791a349b1c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dagster_rocky-1.61.0-py3-none-any.whl:

Publisher: dagster-release.yml on rocky-data/rocky

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

1.61.1

This release

1.61.0

1.60.0

1.59.0

1.58.0

1.57.0

1.56.0

1.55.0

1.54.1

1.54.0

1.53.0

1.52.0

1.51.0

1.50.0

1.49.0

1.48.0

1.47.0

1.46.1

1.46.0

1.45.0

1.44.0

1.43.0

1.42.0

1.41.0

1.40.0

1.39.0

1.38.1

1.38.0

1.37.0

1.36.0

1.35.0

1.34.0

1.33.0

1.32.0

1.31.0

1.30.0

1.29.0

1.28.0

1.27.0

1.26.0

1.25.0

1.24.0

1.23.0

1.22.0

1.21.0

1.20.0

1.19.0

1.18.1

1.18.0

1.17.0

1.16.0

1.15.0

1.14.2

1.14.1

1.14.0

1.13.0

1.12.0

1.11.0

1.10.0

1.9.0

1.8.0

1.7.0

1.6.0

1.5.0

1.4.0

1.3.0

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.1

1.1.0

1.0.0

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page