idg-metadata-client 1.0.2.0

Ingestion Framework for OpenMetadata

Metadata Client

Metadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata.

Usage

Install the package:

pip install metadata-client

Setup a metadata client:

from metadata.ingestion.ometa.ometa_api import OpenMetadata
from metadata.generated.schema.entity.services.connections.metadata.openMetadataConnection import OpenMetadataConnection
from metadata.generated.schema.security.client.openMetadataJWTClientConfig import OpenMetadataJWTClientConfig

server_config = OpenMetadataConnection(
    hostPort="<metadata-server>/api",
    authProvider="openmetadata",
    securityConfig=OpenMetadataJWTClientConfig(
        jwtToken="<your-token>"
    ),
)
client = OpenMetadata(server_config)

When developping in local (only), you may need to accept insecure transport (i.e. http). This is NOT recommanded outside of local development environement, that is requesting localhost.

import os

os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'

Quickstart

Create or update metadata classification:

# Create or update classification
classification_request = CreateClassificationRequest(
    name="TestClassification",
    displayName="TestClassification-2",
    description="Sample classification: {}".format(uuid.uuid4()),
)

client.create_or_update(classification_request)

Documentation

• Service of Assets Management
• Service of Data Building
• Directory of Data Platform

Contributing

Before committing to this repository, you must have pre-commit installed, and install the following pre-commit hooks:

pre-commit install --install-hooks -t pre-commit -t pre-push

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details