DataHub SDK and CLI

DataHub's ingestion framework and CLI — pull metadata from 50+ data sources into your catalog, or push it programmatically from your own pipelines and applications.

Pull-based integrations crawl your data systems on a schedule: Snowflake, BigQuery, dbt, Looker, Airflow, and many more.

Push-based integrations let you emit metadata directly from code as it happens: Python SDK, Java SDK, Spark, Great Expectations, and others.

What you can do

Pull metadata from databases, warehouses, BI tools, and orchestrators using 50+ ready-made connectors
Push metadata programmatically using the Python or Java SDK
Transform and filter metadata in transit using built-in transformers
Schedule and manage ingestion pipelines via CLI or the DataHub UI
Automate lineage, ownership, tags, and documentation across your data assets

Supported sources

Snowflake · BigQuery · Redshift · dbt · Databricks · Looker · Tableau · Power BI · Airflow · Spark · Kafka · PostgreSQL · MySQL · Hive · Glue · S3 · Iceberg · Unity Catalog · Sigma · Mode · Superset · Metabase · and many more

Installation

pip install acryl-datahub
datahub version

Quickstart

Pull metadata from a source (recipe)

# snowflake_recipe.yml
source:
  type: snowflake
  config:
    account_id: my_account
    username: my_user
    password: my_password
    role: DATAHUB_ROLE
    warehouse: COMPUTE_WH

sink:
  type: datahub-rest
  config:
    server: http://localhost:8080

datahub ingest -c snowflake_recipe.yml

Emit metadata from code (SDK)

from datahub.sdk import DataHubClient, Dataset

client = DataHubClient.from_env()

dataset = Dataset(platform="snowflake", name="mydb.schema.table")
dataset.set_description("My table description")
dataset.set_owners(["urn:li:corpuser:jane"])

client.entities.upsert(dataset)

Ingest via the DataHub UI

No CLI required — configure and run ingestion directly from the DataHub UI under Ingestion → Create new source.

Ingestion methods

Method	Best for
CLI + YAML recipe	Scheduled batch ingestion, CI/CD pipelines
Python SDK	Programmatic or event-driven metadata emission
Java SDK	JVM-based integrations (Spark, Flink, etc.)
UI Ingestion	One-click setup, no code required

Key CLI commands

datahub init                          # Connect to a DataHub instance (interactive)
datahub init --username datahub --password datahub  # Quickstart with local defaults
datahub ingest -c recipe.yml          # Run an ingestion pipeline
datahub search "my table"             # Search for entities by keyword
datahub search "*" --filter platform=snowflake --filter entity_type=dataset
datahub graphql --list-operations     # Explore all available GraphQL operations
datahub graphql --query "{ me { corpUser { urn } } }"  # Run a GraphQL query
datahub get --urn <urn>               # Fetch metadata for an entity
datahub delete --urn <urn>            # Delete a metadata entity
datahub timeline --urn <urn>          # View metadata change history

Links

base abs abs-slim aerospike airbyte athena aws-secret-manager azure-ad azure-auth azure-data-factory bigid bigquery bigquery-queries bigquery-slim cassandra circuit-breaker clickhouse clickhouse-usage cockroachdb confluence cube databricks datahub datahub-business-glossary datahub-debug datahub-documents datahub-gc datahub-kafka datahub-lineage-file datahub-lite datahub-pg-queue datahub-rest dataplex db2 dbt dbt-cloud debug-recording delta-lake dlt doris dremio druid dynamodb elasticsearch excel fabric-data-factory fabric-onelake feast fivetran flink gcp-secret-manager gcs gcs-slim github-documents glue grafana hana hex hive hive-metastore iceberg iceberg-catalog informatica informix json-schema kafka kafka-connect kinesis ldap looker lookml mariadb matillion-dpc metabase microstrategy mlflow mode mongodb montecarlo mssql mssql-odbc mysql neo4j nifi notion odcs okta omni onnx-embeddings oracle pinecone postgres powerbi powerbi-report-server preset presto presto-on-hive pulsar qlik-sense quicksight rdf redash redshift redshift-slim s3 s3-slim sac sagemaker salesforce sap-datasphere sigma slack snaplogic snowflake snowflake-queries snowflake-slim snowflake-summary snowplow sql-parser sql-queries sqlalchemy starburst-trino-usage starrocks superset sync-file-emitter tableau teradata thoughtspot tidb timescaledb trino unity-catalog unstructured vertexai vertica all cloud dev docs lint testing-utils integration-tests debug

Release files for acryl-datahub 1.7.0.11rc3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for acryl-datahub 1.7.0.11rc3
File	Size	Uploaded
acryl_datahub-1.7.0.11rc3.tar.gz	4.6 MB	Sep 17, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for acryl-datahub 1.7.0.11rc3
File	Interpreter	ABI	Platform	Reset
acryl_datahub-1.7.0.11rc3-py3-none-any.whl 5.5 MB Sep 17, 2026	Python 3	none	any	Details

Total release size: 10.1 MB

Release files / acryl_datahub-1.7.0.11rc3.tar.gz

Download URL	acryl_datahub-1.7.0.11rc3.tar.gz
Size	4.6 MB
Tags	Source
SHA-256 checksum How to use checksums	`ec3d6fe9525b7d8002be3d90e44d9dd6df565d58a6f4098bb35eb475229b0399`
BLAKE2b-256 checksum How to use checksums	`389eda94458014ebd03cc50d37f97d6503cc262ef2e07a7a3ae2a41218a4d551`
Upload date	Sep 17, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.10.21`

Release files / acryl_datahub-1.7.0.11rc3-py3-none-any.whl

Download URL	acryl_datahub-1.7.0.11rc3-py3-none-any.whl
Size	5.5 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`b48034e7ec08a5b949c2b1ce596fe3b887fe6ed6f139f095539cfd3d462c95e4`
BLAKE2b-256 checksum How to use checksums	`debf245054408d5ebb428fd06d20b2b5d0b47e743b001301c6f31df5580d08af`
Upload date	Sep 17, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.10.21`