DataHub Dagster Plugin
Automatic lineage and run metadata from Dagster into DataHub — captures asset definitions, job runs, and dataset-level lineage with no manual instrumentation.
What you can do
- Capture asset lineage — automatically extract upstream/downstream relationships between Dagster assets and external datasets
- Track run history — record job execution status and task-level outcomes in DataHub
- Map Dagster assets to DataHub entities — assets appear as datasets in your DataHub catalog
- Works with any DataHub deployment — self-hosted or DataHub Cloud
Installation
pip install acryl-datahub-dagster-plugin
Quickstart
Add the DataHub sensor to your Dagster project:
from datahub.ingestion.graph.config import DatahubClientConfig
from datahub_dagster_plugin.sensors.datahub_sensors import (
DatahubDagsterSourceConfig,
make_datahub_sensor,
)
config = DatahubDagsterSourceConfig(
datahub_client_config=DatahubClientConfig(server="http://localhost:8080"),
dagster_url="http://localhost:3000",
)
datahub_sensor = make_datahub_sensor(config=config)
Register it alongside your jobs and assets in your Definitions:
from dagster import Definitions
defs = Definitions(
assets=[...],
jobs=[...],
sensors=[datahub_sensor],
)
Once the sensor is running, every job run will emit lineage and status metadata to DataHub automatically.
Links
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file acryl_datahub_dagster_plugin-1.7.0.3.tar.gz.
File metadata
- Download URL: acryl_datahub_dagster_plugin-1.7.0.3.tar.gz
- Upload date:
- Size: 21.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.10.20
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4d70dce15397bd640d902a15561ef879d807c081cb7448eb04c6bdfb14f5ec3b
|
|
| MD5 |
a180785dcd43e7da30aa0f560bb707c9
|
|
| BLAKE2b-256 |
dba40e0bb8010140d0f79533759d21d5bf5ffdb01ab47931321973a02882fc3d
|
File details
Details for the file acryl_datahub_dagster_plugin-1.7.0.3-py3-none-any.whl.
File metadata
- Download URL: acryl_datahub_dagster_plugin-1.7.0.3-py3-none-any.whl
- Upload date:
- Size: 21.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.10.20
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e9e26b3e8e65f352d7ed2d373eb2fe8a041afef665933a4ddc6ef77bc9a56aac
|
|
| MD5 |
612839e1982ae92720a4a881577e413f
|
|
| BLAKE2b-256 |
4da8cca4ec6f07d3ea5ecf1c55e477d8c54d77653eace51e2893a48190f3e66b
|
