DataHub SDK and CLI

DataHub's ingestion framework and CLI — pull metadata from 50+ data sources into your catalog, or push it programmatically from your own pipelines and applications.

Pull-based integrations crawl your data systems on a schedule: Snowflake, BigQuery, dbt, Looker, Airflow, and many more.

Push-based integrations let you emit metadata directly from code as it happens: Python SDK, Java SDK, Spark, Great Expectations, and others.

What you can do

Pull metadata from databases, warehouses, BI tools, and orchestrators using 50+ ready-made connectors
Push metadata programmatically using the Python or Java SDK
Transform and filter metadata in transit using built-in transformers
Schedule and manage ingestion pipelines via CLI or the DataHub UI
Automate lineage, ownership, tags, and documentation across your data assets

Supported sources

Snowflake · BigQuery · Redshift · dbt · Databricks · Looker · Tableau · Power BI · Airflow · Spark · Kafka · PostgreSQL · MySQL · Hive · Glue · S3 · Iceberg · Unity Catalog · Sigma · Mode · Superset · Metabase · and many more

Installation

pip install acryl-datahub
datahub version

Quickstart

Pull metadata from a source (recipe)

# snowflake_recipe.yml
source:
  type: snowflake
  config:
    account_id: my_account
    username: my_user
    password: my_password
    role: DATAHUB_ROLE
    warehouse: COMPUTE_WH

sink:
  type: datahub-rest
  config:
    server: http://localhost:8080

datahub ingest -c snowflake_recipe.yml

Emit metadata from code (SDK)

from datahub.sdk import DataHubClient, Dataset

client = DataHubClient.from_env()

dataset = Dataset(platform="snowflake", name="mydb.schema.table")
dataset.set_description("My table description")
dataset.set_owners(["urn:li:corpuser:jane"])

client.entities.upsert(dataset)

Ingest via the DataHub UI

No CLI required — configure and run ingestion directly from the DataHub UI under Ingestion → Create new source.

Ingestion methods

Method	Best for
CLI + YAML recipe	Scheduled batch ingestion, CI/CD pipelines
Python SDK	Programmatic or event-driven metadata emission
Java SDK	JVM-based integrations (Spark, Flink, etc.)
UI Ingestion	One-click setup, no code required

Key CLI commands

datahub init                          # Connect to a DataHub instance (interactive)
datahub init --username datahub --password datahub  # Quickstart with local defaults
datahub ingest -c recipe.yml          # Run an ingestion pipeline
datahub search "my table"             # Search for entities by keyword
datahub search "*" --filter platform=snowflake --filter entity_type=dataset
datahub graphql --list-operations     # Explore all available GraphQL operations
datahub graphql --query "{ me { corpUser { urn } } }"  # Run a GraphQL query
datahub get --urn <urn>               # Fetch metadata for an entity
datahub delete --urn <urn>            # Delete a metadata entity
datahub timeline --urn <urn>          # View metadata change history

Links

base abs abs-slim aerospike airbyte athena aws-secret-manager azure-ad azure-auth azure-data-factory bigid bigquery bigquery-queries bigquery-slim cassandra circuit-breaker clickhouse clickhouse-usage cockroachdb confluence cube databricks datahub datahub-business-glossary datahub-debug datahub-documents datahub-gc datahub-kafka datahub-lineage-file datahub-lite datahub-pg-queue datahub-rest dataplex db2 dbt dbt-cloud debug-recording delta-lake dlt doris dremio druid dynamodb elasticsearch excel fabric-data-factory fabric-onelake feast fivetran flink gcp-secret-manager gcs gcs-slim github-documents glue grafana hana hex hive hive-metastore iceberg iceberg-catalog informatica informix json-schema kafka kafka-connect kinesis ldap looker lookml mariadb matillion-dpc metabase microstrategy mlflow mode mongodb montecarlo mssql mssql-odbc mysql neo4j nifi notion odcs okta omni onnx-embeddings oracle pinecone postgres powerbi powerbi-report-server preset presto presto-on-hive pulsar qlik-sense quicksight rdf redash redshift redshift-slim s3 s3-slim sac sagemaker salesforce sap-datasphere sigma slack snaplogic snowflake snowflake-queries snowflake-slim snowflake-summary snowplow sql-parser sql-queries sqlalchemy starburst-trino-usage starrocks superset sync-file-emitter tableau teradata thoughtspot tidb timescaledb trino unity-catalog unstructured vertexai vertica all cloud dev docs lint testing-utils integration-tests debug

Release files for acryl-datahub 1.7.0.11rc4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for acryl-datahub 1.7.0.11rc4
File	Size	Uploaded
acryl_datahub-1.7.0.11rc4.tar.gz	4.6 MB	Sep 17, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for acryl-datahub 1.7.0.11rc4
File	Interpreter	ABI	Platform	Reset
acryl_datahub-1.7.0.11rc4-py3-none-any.whl 5.5 MB Sep 17, 2026	Python 3	none	any	Details

Total release size: 10.1 MB

Release files / acryl_datahub-1.7.0.11rc4.tar.gz

Download URL	acryl_datahub-1.7.0.11rc4.tar.gz
Size	4.6 MB
Tags	Source
SHA-256 checksum How to use checksums	`fe7103fc93289d3a6b3e92a908f819244ea84ce384fc88abdb2e5966d5d0fb00`
BLAKE2b-256 checksum How to use checksums	`d7be0b6bd98a146edf8571ca3a3a6c941a9539680b51d388a83a0cf423d1c331`
Upload date	Sep 17, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.10.21`

Release files / acryl_datahub-1.7.0.11rc4-py3-none-any.whl

Download URL	acryl_datahub-1.7.0.11rc4-py3-none-any.whl
Size	5.5 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`708d9686dbeaac1bab87b5b35f9bab267b632467c744fbd37cfe06901312bcfb`
BLAKE2b-256 checksum How to use checksums	`0a7622b8660e57b52d805b9bd44f758d30b29c206578b05272249bf164f66a04`
Upload date	Sep 17, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.10.21`