— This guide will help you setup the Ingestion framework and connectors —

![Python version 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)

OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites

  • Python >= 3.9.x

### Docs

Please refer to the documentation here https://docs.open-metadata.org/connectors

<img referrerpolicy=”no-referrer-when-downgrade” src=”https://static.scarf.sh/a.png?x-pxid=c1a30c7c-6dc7-4928-95bf-6ee08ca6aa6a” />

### TopologyRunner

All the Ingestion Workflows run through the TopologyRunner.

The flow is depicted in the images below.

TopologyRunner Standard Flow

![image](https://raw.githubusercontent.com/open-metadata/docs-v1/refs/heads/main/public/images/v1.9/features/ingestion/workflows/metadata/multithreading/single-thread-flow.png)

TopologyRunner Multithread Flow

![image](https://raw.githubusercontent.com/open-metadata/docs-v1/refs/heads/main/public/images/v1.9/features/ingestion/workflows/metadata/multithreading/multi-thread-flow.png)

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.0.5.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openmetadata from gravatar.com openmetadata

Credits

Author: OpenMetadata Committers

License

Collate Community License Agreement

Requires

Python >=3.10

Provides Extra

airflow all all-dev-env amundsen athena atlas azure-sso azuresql backup bigquery bigtable cassandra clickhouse cockroach couchbase dagster data-insight databricks datalake-azure datalake-gcs datalake-s3 db2 db2-ibmi dbt deltalake deltalake-spark deltalake-storage dev docs domo doris druid dynamodb e2e-test elasticsearch exasol exasol-test gcs glue googledrive great-expectations greenplum hive impala iomete kafka kafkaconnect kinesis looker mlflow mongo mssql mssql-odbc mysql nifi openlineage opensearch oracle pandas pgspider pii-processor pinotdb playwright postgres powerbi prefect presidio-analyzer presto pubsub pyarrow pymssql qliksense questdb quicksight redash redpanda redshift s3 sagemaker salesforce sample-data sap-hana sas sftp singlestore sklearn slim snowflake ssrs starrocks superset tableau teradata test test-unit trino vertica

Report project as malware

Metadata

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.0.5.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openmetadata from gravatar.com openmetadata

Credits

Author: OpenMetadata Committers

License

Collate Community License Agreement

Requires

Python >=3.10

Provides Extra

airflow all all-dev-env amundsen athena atlas azure-sso azuresql backup bigquery bigtable cassandra clickhouse cockroach couchbase dagster data-insight databricks datalake-azure datalake-gcs datalake-s3 db2 db2-ibmi dbt deltalake deltalake-spark deltalake-storage dev docs domo doris druid dynamodb e2e-test elasticsearch exasol exasol-test gcs glue googledrive great-expectations greenplum hive impala iomete kafka kafkaconnect kinesis looker mlflow mongo mssql mssql-odbc mysql nifi openlineage opensearch oracle pandas pgspider pii-processor pinotdb playwright postgres powerbi prefect presidio-analyzer presto pubsub pyarrow pymssql qliksense questdb quicksight redash redpanda redshift s3 sagemaker salesforce sample-data sap-hana sas sftp singlestore sklearn slim snowflake ssrs starrocks superset tableau teradata test test-unit trino vertica

Report project as malware

Release files for openmetadata-ingestion 2.0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for openmetadata-ingestion 2.0.4.0
File Size Uploaded
openmetadata_ingestion-2.0.4.0.tar.gz 21.1 MB Details

Built distribution (wheel)

Table of built distributions (wheels) for openmetadata-ingestion 2.0.4.0
File Interpreter ABI Platform
openmetadata_ingestion-2.0.4.0-py3-none-any.whl 23.0 MB Python 3 none any Details

Total release size: 44.1 MB

Release files / openmetadata_ingestion-2.0.4.0.tar.gz

Download URL openmetadata_ingestion-2.0.4.0.tar.gz
Size 21.1 MB
Tags Source
SHA-256 checksum
How to use checksums 		5f149d17d161a1bdf890975a2c7a500f41cb623bfc06aa07c10759c2d08b204a
BLAKE2b-256 checksum
How to use checksums 		6c56b58351fdd125629c35be43830e020b719631f375ce1fe39811b372b88b8a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/7.0.0 CPython/3.10.21

Release files / openmetadata_ingestion-2.0.4.0-py3-none-any.whl

Download URL openmetadata_ingestion-2.0.4.0-py3-none-any.whl
Size 23.0 MB
Tags Python 3
SHA-256 checksum
How to use checksums 		5fb8ec8257df2067b6f351103ff464c44903c8af0d0184093d875530f11f4f59
BLAKE2b-256 checksum
How to use checksums 		8298ead8df43e4df066d01adac25c0752958b58e9a2994ce5888e088b95617f7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/7.0.0 CPython/3.10.21

Release history Release notifications | RSS feed

2.0.5.1

2 release files

2.0.5.0

2 release files

This release

2.0.4.0 This release

2 release files

2.0.3.0

2 release files

2.0.2.0

2 release files

2.0.1.0

2 release files

2.0.0.0

2 release files

Pre-release

2.0.0.0rc2

2 release files

Pre-release

2.0.0.0rc1

2 release files

1.13.6.3

2 release files

1.13.6.2

2 release files

1.13.6.1

2 release files

1.13.6.0

2 release files

1.13.5.1

2 release files

1.13.5.0

2 release files

1.13.4.1

2 release files

1.13.4.0

2 release files

1.13.3.3

2 release files

1.13.3.2

2 release files

1.13.3.1

2 release files

1.13.3.0

2 release files

1.13.2.0

2 release files

1.13.1.1

2 release files

1.13.1.0

2 release files

1.13.0.0

2 release files

Pre-release

1.13.0.0rc1

2 release files

1.12.14.2

2 release files

1.12.14.1

2 release files

1.12.14.0

2 release files

1.12.13.0

2 release files

1.12.12.0

2 release files

1.12.11.2

2 release files

1.12.11.1

2 release files

1.12.11.0

2 release files

1.12.10.1

2 release files

1.12.10.0

2 release files

1.12.9.0

2 release files

1.12.8.10

2 release files

1.12.8.9

2 release files

1.12.8.8

2 release files

1.12.8.7

2 release files

1.12.8.6

2 release files

1.12.8.4

2 release files

1.12.8.3

2 release files

1.12.8.2

2 release files

1.12.8.1

2 release files

1.12.8.0

2 release files

1.12.7.0

2 release files

1.12.6.4

2 release files

1.12.6.3

2 release files

1.12.6.2

2 release files

1.12.6.1

2 release files

1.12.6.0

2 release files

1.12.5.6

2 release files

1.12.5.5

2 release files

1.12.5.4

2 release files

1.12.5.3

2 release files

1.12.5.2

2 release files

1.12.5.1

2 release files

1.12.5.0

2 release files

1.12.4.1

2 release files

1.12.4.0

2 release files

1.12.3.0

2 release files

1.12.2.0

2 release files

1.12.1.0

2 release files

1.12.0.0

2 release files

Pre-release

1.12.0.0rc1

2 release files

1.11.14.0

2 release files

1.11.13.0

2 release files

1.11.12.2

2 release files

1.11.12.1

2 release files

1.11.12.0

2 release files

1.11.11.3

2 release files

1.11.11.2

2 release files

1.11.11.1

2 release files

1.11.11.0

2 release files

1.11.10.0

2 release files

1.11.9.2

2 release files

1.11.9.1

2 release files

1.11.9.0

2 release files

1.11.8.1

2 release files

1.11.8.0

2 release files

1.11.7.2

2 release files

1.11.7.1

2 release files

1.11.7.0

2 release files

1.11.6.3

2 release files

1.11.6.2

2 release files

1.11.6.1

2 release files

1.11.6.0

2 release files

1.11.5.1

2 release files

1.11.5.0

2 release files

1.11.4.0

2 release files

1.11.3.0

2 release files

1.11.2.0

2 release files

1.11.1.0

2 release files

1.11.0.0

2 release files

Pre-release

1.11.0.0rc1

2 release files

1.10.14.7

2 release files

1.10.14.6

2 release files

1.10.14.5

2 release files

1.10.14.4

2 release files

1.10.14.3

2 release files

1.10.14.2

2 release files

1.10.14.1

2 release files

1.10.14.0

2 release files

1.10.13.0

2 release files

1.10.12.0

2 release files

1.10.11.2

2 release files

1.10.11.1

2 release files

1.10.11.0

2 release files

1.10.10.0

2 release files

1.10.9.1

2 release files

1.10.9.0

2 release files

1.10.8.0

2 release files

1.10.7.0

2 release files

1.10.6.0

2 release files

1.10.5.0

2 release files

1.10.4.1

2 release files

1.10.4.0

2 release files

1.10.3.0

2 release files

1.10.2.0

2 release files

1.10.1.0

2 release files

1.10.0.0

2 release files

1.9.17.6

2 release files

1.9.17.5

2 release files

1.9.17.4

2 release files

1.9.17.3

2 release files

1.9.17.2

2 release files

1.9.17.1

2 release files

1.9.17.0

2 release files

1.9.16.2

2 release files

1.9.16.1

2 release files

1.9.16.0

2 release files

1.9.15.0

2 release files

1.9.14.1

2 release files

1.9.14.0

2 release files

1.9.13.1

2 release files

Yanked

1.9.13.0

2 release files

Yanked reason: Invalid Release

1.9.12.0

2 release files

1.9.11.9

2 release files

1.9.11.8.1

2 release files

1.9.11.8.0

2 release files

1.9.11.7.0

2 release files

1.9.11.6

2 release files

1.9.11.5

2 release files

1.9.11.4

2 release files

1.9.11.3

2 release files

1.9.11.2

2 release files

1.9.11.1

2 release files

1.9.11.0

2 release files

1.9.10.0

2 release files

1.9.9.1

2 release files

1.9.9.0

2 release files

1.9.8.0

2 release files

1.9.7.1

2 release files

1.9.7.0

2 release files

1.9.6.0

2 release files

1.9.5.0

2 release files

1.9.4.0

2 release files

1.9.3.0

2 release files

1.9.2.1

2 release files

1.9.2.0

2 release files

1.9.1.0

2 release files

1.9.0.0

2 release files

1.8.12.0

2 release files

1.8.11.0

2 release files

1.8.10.0

2 release files

1.8.9.0

2 release files

1.8.8.1

2 release files

1.8.8.0

2 release files

1.8.7.5

2 release files

1.8.7.4

2 release files

1.8.7.3

2 release files

1.8.7.2

2 release files

1.8.7.1

2 release files

1.8.7.0

2 release files

1.8.6.1

2 release files

1.8.6.0

2 release files

1.8.5.0

2 release files

1.8.4.0

2 release files

1.8.3.1

2 release files

1.8.3.0

2 release files

1.8.2.0

2 release files

1.8.1.0

2 release files

1.8.0.0

2 release files

1.7.7.4

2 release files

1.7.7.3

2 release files

1.7.7.2

2 release files

1.7.7.1

2 release files

1.7.7.0

2 release files

1.7.6.0

2 release files

1.7.5.4

2 release files

1.7.5.3

2 release files

1.7.5.2

2 release files

1.7.5.1

2 release files

1.7.5.0

2 release files

1.7.4.2

2 release files

1.7.4.1

2 release files

1.7.4.0

2 release files

1.7.3.0

2 release files

1.7.2.1

2 release files

1.7.2.0

2 release files

1.7.1.5

2 release files

1.7.1.4

2 release files

1.7.1.3

2 release files

1.7.1.2

2 release files

1.7.1.1

2 release files

1.7.1.0

2 release files

1.7.0.1

2 release files

1.7.0.0

2 release files

Pre-release

1.7.0.0rc2

2 release files

Pre-release

1.7.0.0rc1

2 release files

1.6.13.2

2 release files

1.6.13.1

2 release files

1.6.13.0

2 release files

1.6.12.0

2 release files

1.6.11.1

2 release files

1.6.11.0

2 release files

1.6.10.0

2 release files

1.6.9.0

2 release files

1.6.8.0

2 release files

1.6.7.3

2 release files

1.6.7.2

2 release files

1.6.7.1

2 release files

1.6.7.0

2 release files

1.6.6.9

2 release files

1.6.6.8

2 release files

1.6.6.7

2 release files

1.6.6.6

2 release files

1.6.6.5

2 release files

1.6.6.4

2 release files

Yanked

1.6.6.3

2 release files

1.6.6.2

2 release files

1.6.6.1

2 release files

1.6.6.0

2 release files

1.6.5.2

2 release files

1.6.5.1

2 release files

1.6.5.0

2 release files

1.6.4.0

2 release files

1.6.3.3

2 release files

1.6.3.2

2 release files

1.6.3.1

2 release files

1.6.3.0

2 release files

1.6.2.2

2 release files

1.6.2.1

2 release files

1.6.2.0

2 release files

1.6.1.0

2 release files

1.6.0.0

2 release files

Pre-release

1.6.0.0rc3

2 release files

Pre-release

1.6.0.0rc2

2 release files

Pre-release

1.6.0.0rc1

2 release files

1.5.15.2

2 release files

1.5.15.1

2 release files

1.5.15.0

2 release files

1.5.14.1

2 release files

1.5.14.0

2 release files

1.5.13.2

2 release files

1.5.13.1

2 release files

1.5.13.0

2 release files

1.5.12.1

2 release files

1.5.12.0

2 release files

1.5.11.0

2 release files

1.5.10.0

2 release files

1.5.9.0

2 release files

1.5.8.0

2 release files

1.5.7.1

2 release files

1.5.7.0

2 release files

1.5.6.0

2 release files

1.5.5.0

2 release files

1.5.4.1

2 release files

1.5.4.0

2 release files

1.5.3.1

2 release files

1.5.3.0

2 release files

1.5.2.0

2 release files

1.5.1.0

2 release files

1.5.0.0

2 release files

Pre-release

1.5.0.0rc2

2 release files

Pre-release

1.5.0.0rc1

2 release files

1.4.8.1

2 release files

1.4.8.0

2 release files

1.4.7.2

2 release files

1.4.7.1

2 release files

1.4.7.0

2 release files

1.4.6.0

2 release files

Pre-release

1.4.6.0rc1

2 release files

1.4.5.0

2 release files

Pre-release

1.4.5.0rc1

2 release files

1.4.4.2

2 release files

1.4.4.1

2 release files

1.4.4.0

2 release files

Pre-release

1.4.4.0rc1

2 release files

1.4.3.3

2 release files

1.4.3.2

2 release files

1.4.3.1

2 release files

1.4.3.0

2 release files

1.4.2.0

2 release files

1.4.1.0

2 release files

1.4.0.1

2 release files

1.4.0.0

2 release files

Pre-release

1.4.0.0rc4

2 release files

Pre-release

1.4.0.0rc3

2 release files

Pre-release

1.4.0.0rc2

2 release files

Pre-release

1.4.0.0rc1

2 release files

1.3.4.0

2 release files

1.3.3.0

2 release files

Pre-release

1.3.3.0rc2

2 release files

Pre-release

1.3.3.0rc1

2 release files

1.3.2.0

2 release files

Pre-release

1.3.2.0rc3

2 release files

Pre-release

1.3.2.0rc2

2 release files

Pre-release

1.3.2.0rc1

2 release files

1.3.1.3

2 release files

1.3.1.2

2 release files

1.3.1.1

2 release files

1.3.1.0

2 release files

1.3.0.1

2 release files

1.3.0.0

2 release files

1.2.5.3

2 release files

1.2.5.2

2 release files

1.2.5.1

2 release files

1.2.5.0

2 release files

1.2.4.7

2 release files

1.2.4.6

2 release files

1.2.4.5

2 release files

1.2.4.4

2 release files

1.2.4.3

2 release files

1.2.4.2

2 release files

1.2.4.1

2 release files

1.2.4.0

2 release files

1.2.3.3

2 release files

1.2.3.2

2 release files

1.2.3.1

2 release files

1.2.3.0

2 release files

1.2.2.2

2 release files

1.2.2.1

2 release files

1.2.2.0

2 release files

1.2.1.1

2 release files

1.2.1.0

2 release files

1.2.0.1

2 release files

1.2.0.0

2 release files

1.1.7.5

2 release files

1.1.7.4

2 release files

1.1.7.3

2 release files

1.1.7.2

2 release files

1.1.7.1

2 release files

1.1.7.0

2 release files

1.1.6.1

2 release files

1.1.6.0

2 release files

1.1.5.0

2 release files

1.1.4.0

2 release files

1.1.3.0

2 release files

1.1.2.0

2 release files

1.1.1.1

2 release files

1.1.1.0

2 release files

1.1.0.5

2 release files

1.1.0.4

3 release files

1.1.0.3

3 release files

1.1.0.2

3 release files

1.1.0.1

3 release files

1.1.0.0

3 release files

1.0.5.1

3 release files

1.0.5.0

3 release files

1.0.4.4

3 release files

1.0.4.3

3 release files

1.0.4.2

3 release files

1.0.4.1

3 release files

1.0.4.0

3 release files

1.0.3.0

3 release files

1.0.2.0

3 release files

1.0.1.3

3 release files

Yanked

1.0.1.2

3 release files

Yanked reason: openmetadata-managed-apis had 504 gateway timeout while publishing. Yanking this version as well as we maintain same versions for both packages.

1.0.1.1

3 release files

1.0.1.0

3 release files

1.0.0.2

3 release files

1.0.0.1

3 release files

1.0.0.0

3 release files

0.13.4.0

3 release files

0.13.3.1

3 release files

0.13.3.0

3 release files

0.13.2.8

3 release files

0.13.2.7

3 release files

0.13.2.6

3 release files

0.13.2.5

3 release files

0.13.2.4

3 release files

0.13.2.3

3 release files

0.13.2.2

3 release files

0.13.2.1

3 release files

0.13.2.0

3 release files

Pre-release

0.13.2.0b0

3 release files

0.13.1.9

3 release files

0.13.1.8

3 release files

0.13.1.7

3 release files

0.13.1.6

3 release files

0.13.1.5

3 release files

0.13.1.4

3 release files

0.13.1.3

3 release files

0.13.1.2

3 release files

0.13.1.1

3 release files

0.13.1.0

3 release files

0.13.0.3

3 release files

0.13.0.1

3 release files

0.13.0.0

3 release files

0.12.3.2

3 release files

0.12.3.1

3 release files

0.12.3.0

3 release files

0.12.2.4

3 release files

0.12.2.3

3 release files

0.12.2.2

3 release files

0.12.2.1

3 release files

0.12.2.0

3 release files

0.12.1.3

3 release files

0.12.1.2

3 release files

0.12.1.1

3 release files

0.12.1.0

3 release files

0.12.0.3

3 release files

0.12.0.2

3 release files

0.12.0.1

3 release files

0.12.0.0

3 release files

0.11.5.3

3 release files

0.11.5.2

3 release files

0.11.5.1

3 release files

0.11.5.0

3 release files

0.11.4.4

3 release files

0.11.4.3

3 release files

0.11.4.2

3 release files

0.11.4.1

3 release files

0.11.4.0

3 release files

0.11.3.1

3 release files

0.11.3.0

3 release files

0.11.2.0

3 release files

0.11.1.1

3 release files

0.11.1.0

3 release files

0.11.0.0

3 release files

0.10.4.1

3 release files

0.10.4.0

3 release files

0.10.3.2

3 release files

0.10.3.1

3 release files

0.10.3.0

3 release files

0.10.2.0

3 release files

0.10.1.1

3 release files

0.10.1

3 release files

0.10.0.1

3 release files

0.10.0

3 release files

0.9.2

3 release files

0.9.1

3 release files

0.9.0

3 release files

0.8.16

3 release files

0.8.15

3 release files

0.8.14

3 release files

0.8.13

3 release files

0.8.12

3 release files

0.8.11

3 release files

0.8.10

3 release files

0.8.8

3 release files

0.8.7

3 release files

0.8.6

3 release files

0.8.4

3 release files

0.8.3

3 release files

0.8.2

3 release files

0.8.1

3 release files

0.8.0

3 release files

0.7.4

2 release files

0.7.3

2 release files

0.7.2

2 release files

0.7.1

3 release files

0.5.0

2 release files

0.4.11

2 release files

0.4.10

2 release files

0.4.9

2 release files

0.4.8

2 release files

0.4.7

2 release files

0.4.5

2 release files

0.4.4

2 release files

0.4.3

2 release files

0.4.2.1

2 release files

0.4.2

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.2

2 release files

0.3.0

2 release files

0.2.4

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page