hera2-sdk 1.11.8.7

Hera2 Python SDK for metadata ingestion and management

Hera2 Python SDK

A modern, fluent Python SDK for OpenMetadata that provides an intuitive API for all operations. Authentication is routed through the Heimdall authorization service for DataOS integration.

Installing hera2-sdk pulls in hera2-ingestion (HERA-modified fork of openmetadata-ingestion) automatically and shares the metadata namespace, so you get both metadata.sdk (Heimdall auth, fluent API) and the full ingestion stack: metadata.ingestion, metadata.generated, metadata.clients, metadata.profiler, metadata.utils, metadata.workflow, etc.

Installation

Python 3.9+ required. We recommend Python 3.10 or 3.11 for best compatibility.

macOS note: On macOS, python3.10 or python3.11 in your shell may be a symlink to the system Python 3.9. Always use the full path to the Homebrew-installed interpreter when creating the venv:

# Install Python 3.11 via Homebrew if not already available
brew install python@3.11

# Verify the Homebrew interpreter (M-series Mac uses /opt/homebrew, Intel uses /usr/local)
/opt/homebrew/bin/python3.11 --version   # should print 3.11.x

# Create the venv using the full path
/opt/homebrew/bin/python3.11 -m venv .venv
source .venv/bin/activate

# Always use python3 -m pip (not bare pip) to avoid shell alias interference
python3 -m pip install hera2-sdk

After activation, confirm the interpreter is correct before installing:

python3 --version   # must show 3.11.x (or 3.10.x)
python3 -m pip --version   # must show python 3.11 (or 3.10), not 3.9

For an editable install from the repo:

pip install -e /path/to/hera2/ingestion/
pip install -e /path/to/hera2/hera2-sdk/

Data Quality SDK Installation

For running data quality tests, additional dependencies may be required:

DataFrame Validation:

pip install 'hera2-sdk[pandas]'

Table-Based Testing:

pip install 'hera2-sdk[mysql]'        # For MySQL
pip install 'hera2-sdk[postgres]'     # For PostgreSQL
pip install 'hera2-sdk[snowflake]'    # For Snowflake
pip install 'hera2-sdk[clickhouse]'   # For ClickHouse

Troubleshooting

• python --version shows 3.9 inside the venv — Your shell has an alias (alias python=...) that overrides venv activation. Check with:

  type python   # shows "python is an alias for ..."
  grep -n "python" ~/.zshrc ~/.zprofile 2>/dev/null
  

  Remove the alias from ~/.zshrc / ~/.zprofile and reload (source ~/.zshrc). Until then, use python3 instead of python — the venv’s python3 is not affected by the alias.

• pip --version shows python 3.9 — Similarly, pip may be aliased to the system pip (alias pip=/usr/bin/pip3). Always use python3 -m pip instead of bare pip to drive pip through the venv interpreter:

  python3 -m pip install hera2-sdk==<version>
  

• "Defaulting to user installation because normal site-packages is not writeable" — Pip is installing to your user directory instead of the active venv.

  1. Use the venv’s Python explicitly: python3 -m pip install hera2-sdk.
  2. If it still happens, make the venv writable and reinstall: chmod -R u+w .venv (or your venv dir), then python3 -m pip install --force-reinstall hera2-sdk.
  3. Confirm the venv is the one being used: python3 -c "import sys; print(sys.prefix)" should print a path inside your venv. If it prints a system path, the venv wasn’t activated or python3 in your shell isn’t from the venv.

• TypeError: unsupported operand type(s) for |: 'type' and 'NoneType' when importing metadata.ingestion — The interpreter is loading the local metadata package from the hera2 ingestion source (.../hera2/ingestion/src/metadata/) instead of the installed hera2-ingestion. Something is adding that path to sys.path.

  1. Unset PYTHONPATH before running Python: unset PYTHONPATH, then run your script again.
  2. See what adds the repo to the path: python3 -c "import sys; print([p for p in sys.path if 'hera2' in p or 'ingestion' in p])". If you see a path like .../hera2/ingestion/src, it was added by PYTHONPATH or a .pth file in site-packages.
  3. Run Python from a directory that is outside the hera2 repo.

• UserWarning: pkg_resources is deprecated as an API on import — Harmless deprecation warning. Suppress with:

  import warnings
  warnings.filterwarnings("ignore", category=UserWarning, module="pkg_resources")
  

  Or via environment variable: PYTHONWARNINGS="ignore::UserWarning:pkg_resources" python3 your_script.py

Quick Start

Configure the SDK (Heimdall Auth — Recommended)

Use heimdallConfiguration (same structure as hera/config/config.yaml authenticationConfiguration.heimdallConfiguration):

from metadata.sdk import configure

configure(
    host="http://localhost:8585/api",
    api_key="your-dataos-api-key",
    heimdall_configuration={
        "enabled": True,
        "baseUrl": "https://your-instance.dataos.cloud/heimdall",
        "timeout": 10,
        "fallbackOnBasic": True,
    },
)

Or use the legacy heimdall_url:

configure(
    host="http://localhost:8585/api",
    api_key="your-dataos-api-key",
    heimdall_url="https://your-instance.dataos.cloud/heimdall",
)

Or set environment variables and call configure() with no arguments:

export OPENMETADATA_HOST="http://localhost:8585/api"
export OPENMETADATA_API_KEY="your-dataos-api-key"
export HEIMDALL_BASE_URL="https://your-instance.dataos.cloud/heimdall"

from metadata.sdk import configure
configure()

Configure Parameters

The configure() function supports:

• host or server_url: OpenMetadata server URL
• api_key or jwt_token: DataOS API key or JWT token
• heimdall_configuration: Dict matching hera/config/config.yaml heimdallConfiguration (enabled, baseUrl, timeout, fallbackOnBasic, trustAll)
• heimdall_url: Heimdall base URL (legacy; use heimdall_configuration when possible)
• Falls back to environment variables:
  • OPENMETADATA_HOST or OPENMETADATA_SERVER_URL for the server URL
  • OPENMETADATA_API_KEY or OPENMETADATA_JWT_TOKEN for authentication
  • HEIMDALL_BASE_URL: Heimdall service URL (enables Heimdall auth)
  • HEIMDALL_TIMEOUT: Heimdall request timeout in seconds (default: 10)
  • HEIMDALL_TRUST_ALL: Trust all SSL certs for Heimdall (default: true)
  • OPENMETADATA_VERIFY_SSL: Enable SSL verification (default: false)
  • OPENMETADATA_CA_BUNDLE: Path to CA bundle
  • OPENMETADATA_CLIENT_TIMEOUT: Client timeout in seconds (default: 30)

Alternative: Builder Pattern

from metadata.sdk.config import OpenMetadataConfig

config = (
    OpenMetadataConfig.builder()
    .server_url("http://localhost:8585/api")
    .api_key("your-dataos-api-key")
    .heimdall_configuration({
        "enabled": True,
        "baseUrl": "https://your-instance.dataos.cloud/heimdall",
        "timeout": 15,
        "fallbackOnBasic": True,
    })
    .build()
)

Or with flat params: .heimdall_url("...").heimdall_timeout(15).

Alternative: Direct JWT (Legacy)

If Heimdall is not available, the SDK falls back to direct JWT authentication:

from metadata.sdk import configure
configure(host="http://localhost:8585/api", jwt_token="your-om-jwt-token")

Using like the OpenMetadata Python SDK

hera2-sdk depends on hera2-ingestion, so you can use the same low-level API as in the official OpenMetadata SDK docs: OpenMetadataConnection + OpenMetadata(server_config), then metadata.create_or_update(), metadata.get_by_name(), metadata.delete(), etc.

Option 1 — Standard OpenMetadata style (same as the docs)

from metadata.ingestion.ometa.ometa_api import OpenMetadata
from metadata.generated.schema.entity.services.connections.metadata.openMetadataConnection import (
    OpenMetadataConnection,
    AuthProvider,
)
from metadata.generated.schema.security.client.openMetadataJWTClientConfig import (
    OpenMetadataJWTClientConfig,
)
from metadata.generated.schema.entity.data.table import Table

server_config = OpenMetadataConnection(
    hostPort="http://localhost:8585/api",
    authProvider=AuthProvider.openmetadata,
    securityConfig=OpenMetadataJWTClientConfig(
        jwtToken="<YOUR-INGESTION-BOT-JWT-TOKEN>",
    ),
)
metadata = OpenMetadata(server_config)

# Same API as in the docs
metadata.health_check()
service_entity = metadata.create_or_update(data=create_service)
my_table = metadata.get_by_name(entity=Table, fqn="test-service-table.test-db.test-schema.test")
metadata.delete(entity=Table, entity_id=my_table.id)

Option 2 — hera2-sdk wrapper (same API + optional Heimdall)

Use configure() or OpenMetadataConfig, then get the underlying client via .ometa and call the same methods:

from metadata.sdk import configure, client
from metadata.generated.schema.entity.data.table import Table

configure(
    host="http://localhost:8585/api",
    api_key="your-dataos-api-key",
    heimdall_configuration={
        "enabled": True,
        "baseUrl": "https://your-instance.dataos.cloud/heimdall",
        "timeout": 10,
        "fallbackOnBasic": True,
    },
)

metadata = client().ometa   # same interface as OpenMetadata(server_config)

metadata.health_check()
service_entity = metadata.create_or_update(data=create_service)
my_table = metadata.get_by_name(entity=Table, fqn="test-service-table.test-db.test-schema.test")
metadata.delete(entity=Table, entity_id=my_table.id)

So you can follow the OpenMetadata SDK walkthrough (create DatabaseService, Database, Schema, Table, etc.) with either the raw OpenMetadata from metadata.ingestion.ometa.ometa_api or with client().ometa after configuring hera2-sdk.

Manual Initialization

For more control, you can manually initialize the SDK:

from metadata.sdk import OpenMetadata, OpenMetadataConfig
from metadata.sdk.entities import Table, User
from metadata.sdk.api import Search, Lineage, Bulk

config = OpenMetadataConfig(
    server_url="http://localhost:8585/api",
    api_key="your-dataos-api-key",
    heimdall_configuration={
        "enabled": True,
        "baseUrl": "https://your-instance.dataos.cloud/heimdall",
        "timeout": 10,
        "fallbackOnBasic": True,
    },
)

client = OpenMetadata.initialize(config)

Table.set_default_client(client)
User.set_default_client(client)
Search.set_default_client(client)
Lineage.set_default_client(client)
Bulk.set_default_client(client)

Configuration from Environment Variables Only

from metadata.sdk.config import OpenMetadataConfig

# Reads from OPENMETADATA_HOST, OPENMETADATA_API_KEY, HEIMDALL_BASE_URL, etc.
config = OpenMetadataConfig.from_env()

Entity Operations

Tables

from metadata.generated.schema.api.data.createTable import CreateTableRequest
from metadata.sdk.entities.table import TableListParams

# Create a table
request = CreateTableRequest(
    name="my_table",
    databaseSchema="my_schema",
    columns=[...]
)
table = Table.create(request)

# Retrieve a table by ID
table = Table.retrieve("table-id")

# Retrieve by fully qualified name with specific fields
table = Table.retrieve_by_name(
    "service.database.schema.table",
    fields=["owners", "tags", "columns"]
)

# List tables with pagination
for table in Table.list().auto_paging_iterable():
    print(table.name)

# List with filters
params = TableListParams.builder() \
    .limit(50) \
    .database("my_database") \
    .fields(["owners", "tags"]) \
    .build()

tables = Table.list(params)

# Update a table
table.description = "Updated description"
updated = Table.update(table.id, table)

# Delete a table
Table.delete("table-id")

# Delete with options
Table.delete("table-id", recursive=True, hard_delete=True)

# Export/Import CSV
csv_data = Table.export_csv("table-name")
Table.import_csv(csv_data, dry_run=False)

Supported Entity Types

The SDK provides the same fluent API for all OpenMetadata entity types:

• Data Assets: Table, Database, DatabaseSchema, Dashboard, Pipeline, Topic, Container, Query, StoredProcedure, DashboardDataModel, SearchIndex, MlModel, Report
• Services: DatabaseService, MessagingService, DashboardService, PipelineService, MlModelService, StorageService, SearchService, MetadataService, ApiService
• Teams & Users: User, Team, Role, Policy
• Governance: Glossary, GlossaryTerm, Classification, Tag, DataProduct, Domain
• Quality: TestCase, TestSuite, TestDefinition, DataQualityDashboard
• Ingestion: Ingestion, Workflow, Connection
• Other: Type, Webhook, Kpi, Application, Persona, DocStore, Page, SearchQuery

Testing

Run the SDK tests:

# Run all SDK tests
pytest tests/unit/sdk/

# Run specific test
pytest tests/unit/sdk/test_sdk_entities.py

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.