Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

DataHub Prefect Plugin

Automatic lineage and run metadata from Prefect into DataHub — captures flow structure, task inputs/outputs, and run history with minimal setup.

What you can do

  • Emit flow and task metadata to DataHub as pipeline runs
  • Capture dataset lineage — declare inputs and outputs per task and see them in DataHub
  • Configure via Prefect blocks — store your DataHub connection settings as a reusable block
  • Works with any DataHub deployment — self-hosted or DataHub Cloud

Installation

pip install prefect-datahub

Quickstart

1. Save your DataHub connection as a Prefect block

from prefect_datahub.datahub_emitter import DatahubEmitter

DatahubEmitter(
    datahub_rest_url="http://localhost:8080",
    env="PROD",
).save("my-datahub")

2. Use it in your flows

from prefect import flow, task
from prefect_datahub.datahub_emitter import DatahubEmitter
from prefect_datahub.entities import Dataset

emitter = DatahubEmitter.load("my-datahub")

@task
def transform(data, emitter):
    emitter.add_task(
        inputs=[Dataset("snowflake", "mydb.schema.source_table")],
        outputs=[Dataset("snowflake", "mydb.schema.output_table")],
    )
    return data

@flow
def my_pipeline():
    data = extract()
    transform(data, emitter)
    emitter.emit_flow()   # required — emits all metadata at the end

Configuration options

Option Default Description
datahub_rest_url http://localhost:8080 DataHub GMS URL
env PROD Environment tag for assets
platform_instance None Platform instance for assets
token None Auth token (if GMS auth is enabled)

Metadata

Release files for prefect-datahub 1.7.0.20rc4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for prefect-datahub 1.7.0.20rc4
File Size Uploaded Provenance Actions
prefect_datahub-1.7.0.20rc4.tar.gz 13.8 kB Without provenance Details for prefect_datahub-1.7.0.20rc4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for prefect-datahub 1.7.0.20rc4
File Interpreter ABI Platform Actions
prefect_datahub-1.7.0.20rc4-py3-none-any.whl Python 3 none any Details for prefect_datahub-1.7.0.20rc4-py3-none-any.whl

Total release size: 25.7 kB

Release files / prefect_datahub-1.7.0.20rc4.tar.gz

Download URL prefect_datahub-1.7.0.20rc4.tar.gz
Size 13.8 kB
Tags Source
SHA-256 checksum
How to use checksums
f77133350f1d8c99919bd3d6b2341fefc6f85a289e4e94816d040ff9e46a403b
BLAKE2b-256 checksum
How to use checksums
d92aeee8cb709f6dd86c744a78368ea92f821f879c388dabd23584c969e6c703
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Release files / prefect_datahub-1.7.0.20rc4-py3-none-any.whl

Download URL prefect_datahub-1.7.0.20rc4-py3-none-any.whl
Size 11.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f6dad63bc89bd63b4390e9051d55aa27dd9fc375bc0e56a1422f86b2531846c3
BLAKE2b-256 checksum
How to use checksums
34ada9b2e2dac73effe1a86475d6ed2c1bb7a95f8a13476627fbeb6f131b1aca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Security for prefect-datahub 1.7.0.20rc4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page