Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

DataHub Prefect Plugin

Automatic lineage and run metadata from Prefect into DataHub — captures flow structure, task inputs/outputs, and run history with minimal setup.

What you can do

  • Emit flow and task metadata to DataHub as pipeline runs
  • Capture dataset lineage — declare inputs and outputs per task and see them in DataHub
  • Configure via Prefect blocks — store your DataHub connection settings as a reusable block
  • Works with any DataHub deployment — self-hosted or DataHub Cloud

Installation

pip install prefect-datahub

Quickstart

1. Save your DataHub connection as a Prefect block

from prefect_datahub.datahub_emitter import DatahubEmitter

DatahubEmitter(
    datahub_rest_url="http://localhost:8080",
    env="PROD",
).save("my-datahub")

2. Use it in your flows

from prefect import flow, task
from prefect_datahub.datahub_emitter import DatahubEmitter
from prefect_datahub.entities import Dataset

emitter = DatahubEmitter.load("my-datahub")

@task
def transform(data, emitter):
    emitter.add_task(
        inputs=[Dataset("snowflake", "mydb.schema.source_table")],
        outputs=[Dataset("snowflake", "mydb.schema.output_table")],
    )
    return data

@flow
def my_pipeline():
    data = extract()
    transform(data, emitter)
    emitter.emit_flow()   # required — emits all metadata at the end

Configuration options

Option Default Description
datahub_rest_url http://localhost:8080 DataHub GMS URL
env PROD Environment tag for assets
platform_instance None Platform instance for assets
token None Auth token (if GMS auth is enabled)

Metadata

Release files for prefect-datahub 1.8.0rc1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for prefect-datahub 1.8.0rc1
File Size Uploaded Provenance Actions
prefect_datahub-1.8.0rc1.tar.gz 13.8 kB Without provenance Details for prefect_datahub-1.8.0rc1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for prefect-datahub 1.8.0rc1
File Interpreter ABI Platform Actions
prefect_datahub-1.8.0rc1-py3-none-any.whl Python 3 none any Details for prefect_datahub-1.8.0rc1-py3-none-any.whl

Total release size: 25.7 kB

Release files / prefect_datahub-1.8.0rc1.tar.gz

Download URL prefect_datahub-1.8.0rc1.tar.gz
Size 13.8 kB
Tags Source
SHA-256 checksum
How to use checksums
b851fcfdeebf9555b4564422d8b9397447cc26f42575be68b29a39813c4dcb8e
BLAKE2b-256 checksum
How to use checksums
6cde8c24a4be1c6fd369a1e6041469f33ecbb10a917c6333f398e36e48d4cef5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Release files / prefect_datahub-1.8.0rc1-py3-none-any.whl

Download URL prefect_datahub-1.8.0rc1-py3-none-any.whl
Size 11.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2c8275cfc55eb39be0e14edac594aa275da583dd758d6a31838010c9f56f3a0b
BLAKE2b-256 checksum
How to use checksums
5e0f2210edb07b13f4efa478417c7be4a6aeab8c203317c0729e34ca9c6cb452
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.22

Security for prefect-datahub 1.8.0rc1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page