iil-enrichment

Enrichment Agent Pattern — bridge managed records with external knowledge sources.

ADR-169: Generic pattern for enriching document-centric records (SDS, GBU, tenders, trades) with data from authoritative external APIs (GESTIS, PubChem, ECHA, market data, etc.)

Installation

# Core only (no providers)
pip install iil-enrichment

# With GESTIS provider
pip install iil-enrichment[gestis]

# With Django integration
pip install iil-enrichment[django]

# Everything
pip install iil-enrichment[all]

Quick Start

from enrichment.registry import EnrichmentRegistry
from enrichment.providers.gestis import GESTISProvider
from enrichment.providers.pubchem import PubChemProvider

# Setup registry
registry = EnrichmentRegistry()
registry.register("substance", GESTISProvider())
registry.register("substance", PubChemProvider())

# Enrich a substance by CAS number
result = registry.enrich_merged("substance", "67-64-1")

# Access structured data
print(result.get("agw"))           # AGW from GESTIS
print(result.get("first_aid"))     # First aid from GESTIS
print(result.get("molecular_formula"))  # Formula from PubChem
print(result.confidence)           # 0.0–1.0

Django Integration

from enrichment.django.mixins import EnrichableModelMixin

class GlobalSdsRevision(EnrichableModelMixin, models.Model):
    substance = models.ForeignKey(...)

    def get_natural_key_for_enrichment(self):
        return self.substance.cas_number or self.product_name

    def get_enrichment_domain(self):
        return "sds"

# Usage
revision.run_enrichment()  # → fetches GESTIS + PubChem, stores in enrichment_data JSONB

Architecture

┌──────────────┐     ┌──────────────────────┐
│ Managed      │     │ EnrichmentRegistry   │
│ Record       │────▶│                      │
│ (DMS-focus)  │     │  ┌──────────────┐   │
│              │     │  │ GESTISProv.  │   │
│ enrichment_  │◀────│  │ PubChemProv. │   │
│ data (JSONB) │     │  │ CustomProv.  │   │
└──────────────┘     │  └──────────────┘   │
                     └──────────────────────┘

Built-in Providers

Configuration via Environment

All settings have sensible defaults and can be overridden per environment:

Or configure programmatically:

from enrichment.config import HTTPDefaults
from enrichment.providers.gestis import GESTISProvider

defaults = HTTPDefaults(timeout_seconds=30, max_retries=5, cache_enabled=True)
provider = GESTISProvider(defaults=defaults)

Custom Provider

from enrichment.provider import EnrichmentProvider
from enrichment.types import EnrichmentResult, PropertyValue

class MyProvider:
    @property
    def name(self) -> str:
        return "MySource"

    @property
    def supported_domains(self) -> list[str]:
        return ["my_domain"]

    def can_enrich(self, domain: str, natural_key: str) -> bool:
        return domain == "my_domain" and bool(natural_key)

    def enrich(self, domain: str, natural_key: str) -> EnrichmentResult:
        # Fetch from your API...
        return EnrichmentResult(
            source=self.name,
            confidence=0.9,
            properties={"key": PropertyValue(value="data")},
            natural_key=natural_key,
        )

Ecosystem

iil-enrichment ist die Anreicherungs-Stufe einer dreiteiligen Datenpipeline:

iil-ingest  →  iil-enrichment  →  iil-fieldprefill
(extract)      (enrich)           (prefill form fields)

Eingang ist ein natural key (z. B. CAS-Nummer, oft aus iil-ingest-Text extrahiert); Ausgang EnrichmentResult.properties speist den Kontext für iil-fieldprefill. Vollständiger Flow, reale Typen und ein lauffähiges End-to-End-Beispiel: platform/docs/ecosystem/iil-data-pipeline.md.

License

MIT