Ingestion Framework for OpenMetadata

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for akshayjain3450 from gravatar.com akshayjain3450

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Collate Community License Agreement
  • Author: OpenMetadata Committers
  • Requires: Python >=3.9
  • Provides-Extra: airflow , all , all-dev-env , amundsen , athena , atlas , azure-sso , azuresql , backup , bigquery , bigtable , cassandra , clickhouse , cockroach , couchbase , dagster , data-diff , data-insight , databricks , datalake-azure , datalake-gcs , datalake-s3 , db2 , db2-ibmi , dbt , deltalake , deltalake-spark , deltalake-storage , dev , docs , domo , doris , druid , dynamodb , e2e-test , elasticsearch , exasol , glue , great-expectations , great-expectations-1xx , greenplum , hive , iceberg , impala , kafka , kafkaconnect , kinesis , looker , mlflow , mongo , mssql , mssql-odbc , mysql , nifi , openlineage , opensearch , oracle , pandas , pgspider , pii-processor , pinotdb , playwright , postgres , powerbi , presidio-analyzer , presto , pyarrow , pymssql , qliksense , quicksight , redash , redpanda , redshift , sagemaker , salesforce , sample-data , sap-hana , sas , singlestore , sklearn , slim , snowflake , superset , tableau , teradata , test , test-unit , trino , vertica
Report project as malware

Project description

— This guide will help you setup the Ingestion framework and connectors —

![Python version 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)

OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites

  • Python >= 3.9.x

### Docs

Please refer to the documentation here https://docs.open-metadata.org/connectors

<img referrerpolicy=”no-referrer-when-downgrade” src=”https://static.scarf.sh/a.png?x-pxid=c1a30c7c-6dc7-4928-95bf-6ee08ca6aa6a” />

### TopologyRunner

All the Ingestion Workflows run through the TopologyRunner.

The flow is depicted in the images below.

TopologyRunner Standard Flow

![image](https://raw.githubusercontent.com/open-metadata/docs-v1/refs/heads/main/public/images/v1.9/features/ingestion/workflows/metadata/multithreading/single-thread-flow.png)

TopologyRunner Multithread Flow

![image](https://raw.githubusercontent.com/open-metadata/docs-v1/refs/heads/main/public/images/v1.9/features/ingestion/workflows/metadata/multithreading/multi-thread-flow.png)

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for akshayjain3450 from gravatar.com akshayjain3450

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Collate Community License Agreement
  • Author: OpenMetadata Committers
  • Requires: Python >=3.9
  • Provides-Extra: airflow , all , all-dev-env , amundsen , athena , atlas , azure-sso , azuresql , backup , bigquery , bigtable , cassandra , clickhouse , cockroach , couchbase , dagster , data-diff , data-insight , databricks , datalake-azure , datalake-gcs , datalake-s3 , db2 , db2-ibmi , dbt , deltalake , deltalake-spark , deltalake-storage , dev , docs , domo , doris , druid , dynamodb , e2e-test , elasticsearch , exasol , glue , great-expectations , great-expectations-1xx , greenplum , hive , iceberg , impala , kafka , kafkaconnect , kinesis , looker , mlflow , mongo , mssql , mssql-odbc , mysql , nifi , openlineage , opensearch , oracle , pandas , pgspider , pii-processor , pinotdb , playwright , postgres , powerbi , presidio-analyzer , presto , pyarrow , pymssql , qliksense , quicksight , redash , redpanda , redshift , sagemaker , salesforce , sample-data , sap-hana , sas , singlestore , sklearn , slim , snowflake , superset , tableau , teradata , test , test-unit , trino , vertica

Release history Release notifications | RSS feed

1.11.8.8

1.11.8.7

1.11.8.6

1.11.8.5

1.11.8.0.1

This version

1.11.8.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hera2_sdk-1.11.8.0.0.tar.gz (20.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

hera2_sdk-1.11.8.0.0-py3-none-any.whl (21.9 MB view details)

Uploaded Python 3

File details

Details for the file hera2_sdk-1.11.8.0.0.tar.gz.

File metadata

  • Download URL: hera2_sdk-1.11.8.0.0.tar.gz
  • Upload date:
  • Size: 20.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.13

File hashes

Hashes for hera2_sdk-1.11.8.0.0.tar.gz
Algorithm Hash digest
SHA256 d3c396263bd88a3898c6acf7db15696cd7e3c76300c6395005f09082f5db3f7e
MD5 71b42004bb79717d0ef0832384a3910a
BLAKE2b-256 4330561a434a75a5233da31cbf79344046adf4a3d35f2ed45798da1b5b06da45

See more details on using hashes here.

File details

Details for the file hera2_sdk-1.11.8.0.0-py3-none-any.whl.

File metadata

  • Download URL: hera2_sdk-1.11.8.0.0-py3-none-any.whl
  • Upload date:
  • Size: 21.9 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.13

File hashes

Hashes for hera2_sdk-1.11.8.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c99e0d1cbd61032f5c25c74f92aceae6feb602419f9276cd487f059465ebd514
MD5 8504e2e2f3a0de602a9004abcf3b0515
BLAKE2b-256 97c9c4b87de5ad6c24f9d74730a5fab359ac894324df5e0797e757d3becf50b4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page