Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

DataHub Prefect Plugin

Automatic lineage and run metadata from Prefect into DataHub — captures flow structure, task inputs/outputs, and run history with minimal setup.

What you can do

  • Emit flow and task metadata to DataHub as pipeline runs
  • Capture dataset lineage — declare inputs and outputs per task and see them in DataHub
  • Configure via Prefect blocks — store your DataHub connection settings as a reusable block
  • Works with any DataHub deployment — self-hosted or DataHub Cloud

Installation

pip install prefect-datahub

Quickstart

1. Save your DataHub connection as a Prefect block

from prefect_datahub.datahub_emitter import DatahubEmitter

DatahubEmitter(
    datahub_rest_url="http://localhost:8080",
    env="PROD",
).save("my-datahub")

2. Use it in your flows

from prefect import flow, task
from prefect_datahub.datahub_emitter import DatahubEmitter
from prefect_datahub.entities import Dataset

emitter = DatahubEmitter.load("my-datahub")

@task
def transform(data, emitter):
    emitter.add_task(
        inputs=[Dataset("snowflake", "mydb.schema.source_table")],
        outputs=[Dataset("snowflake", "mydb.schema.output_table")],
    )
    return data

@flow
def my_pipeline():
    data = extract()
    transform(data, emitter)
    emitter.emit_flow()   # required — emits all metadata at the end

Configuration options

Option Default Description
datahub_rest_url http://localhost:8080 DataHub GMS URL
env PROD Environment tag for assets
platform_instance None Platform instance for assets
token None Auth token (if GMS auth is enabled)

Metadata

Release files for prefect-datahub 1.7.0.20rc5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for prefect-datahub 1.7.0.20rc5
File Size Uploaded Provenance Actions
prefect_datahub-1.7.0.20rc5.tar.gz 13.8 kB Without provenance Details for prefect_datahub-1.7.0.20rc5.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for prefect-datahub 1.7.0.20rc5
File Interpreter ABI Platform Actions
prefect_datahub-1.7.0.20rc5-py3-none-any.whl Python 3 none any Details for prefect_datahub-1.7.0.20rc5-py3-none-any.whl

Total release size: 25.8 kB

Release files / prefect_datahub-1.7.0.20rc5.tar.gz

Download URL prefect_datahub-1.7.0.20rc5.tar.gz
Size 13.8 kB
Tags Source
SHA-256 checksum
How to use checksums
cc9d7bcb5d94a4282245aeb87c1114779619bf0d544aa92c47bd0717cd5833be
BLAKE2b-256 checksum
How to use checksums
4f1d0b3ffa4795d49de318e6fa427f773778d661484147b78981793c71ca1ced
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Release files / prefect_datahub-1.7.0.20rc5-py3-none-any.whl

Download URL prefect_datahub-1.7.0.20rc5-py3-none-any.whl
Size 12.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
578400ae23fd9c6088c455f9027e64417b1c03cd06b6958580aa761033a03bd0
BLAKE2b-256 checksum
How to use checksums
0421b7771f5f19f549d942919117f49873b1501ae14b8010c8c3eec103574dd7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Security for prefect-datahub 1.7.0.20rc5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page