helmric 0.1.0

Audit + reliability layer for AI agents. One call. Zero hallucinations in production.

helmric

Audit + reliability layer for AI agents.

One SDK call wraps any LLM output, cross-references every cited ID against the source data you gave the model, drops hallucinations, and writes an append-only audit trail.

---

Install

pip install helmric

Quickstart

from helmric import audit

result = audit.verify(
    output=llm_response_text,
    sources=[
        {"id": "user_123", "name": "Anna"},
        {"id": "user_456", "name": "Ben"},
    ],
    api_key="hk_live_...",   # or set HELMRIC_API_KEY env var
)

print(result.cleaned_output)   # LLM output with hallucinated IDs removed
print(result.hallucinated)     # ["d4e5f6a7-..."]  IDs the model invented
print(result.cited)            # ["user_123"]       IDs that actually grounded
print(result.audit_id)         # UUID of the audit row
print(result.audit_url)        # "https://helmric.com/audit/<id>"

API Key

Three resolution sources, in priority order:

1. Explicit api_key= kwarg
2. HELMRIC_API_KEY environment variable ← recommended for production
3. ~/.helmric/config.toml with api_key = "hk_live_..."

export HELMRIC_API_KEY="hk_live_..."

Async

import asyncio
from helmric import audit

async def main():
    result = await audit.verify_async(
        output=llm_response,
        sources=[{"id": "doc_42"}],
    )
    print(result.hallucinated)

asyncio.run(main())

Error handling

from helmric import audit
from helmric.errors import HelmricAuthError, HelmricRateLimitError, HelmricError
import time

try:
    result = audit.verify(output=response, sources=sources)
except HelmricAuthError:
    print("Bad or revoked API key — check your HELMRIC_API_KEY")
except HelmricRateLimitError as e:
    time.sleep(e.retry_after)
    result = audit.verify(output=response, sources=sources)
except HelmricError as e:
    print(f"API error {e.status_code}: {e.detail}")

Parameters

audit.verify() / audit.verify_async()

Parameter	Type	Required	Description
output	str	✓	Raw LLM response text
sources	list[dict]	✓	Source objects given to the model. Each must have an id key.
api_key	str	—	Helmric API key. Falls back to env var.
base_url	str	—	Override API base URL (for self-hosted).
model_provider	str	—	e.g. "openai". Stored in audit trail.
model_name	str	—	e.g. "gpt-4o". Stored in audit trail.
customer_request_id	str	—	Your correlation ID. Stored in audit trail.
timeout	float	—	Request timeout in seconds. Default 30.

VerifyResult fields

Field	Type	Description
cleaned_output	str	LLM output with hallucinated IDs removed
hallucinated	list[str]	IDs the model invented (not in sources)
cited	list[str]	IDs the model correctly cited (in sources)
audit_id	uuid.UUID	UUID of the append-only audit row
audit_url	str	Permalink to the audit event in the dashboard

Real-world example

import openai
from helmric import audit

client = openai.OpenAI()
sources = fetch_users_from_db(org_id)   # your data

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Summarize team performance."},
        {"role": "user", "content": format_sources(sources)},
    ],
)
llm_text = response.choices[0].message.content

result = audit.verify(
    output=llm_text,
    sources=[{"id": str(u.id), "name": u.name} for u in sources],
    model_provider="openai",
    model_name="gpt-4o",
)

# Use result.cleaned_output in your product — hallucinations scrubbed.
# result.audit_url gives you a permanent link to the decision audit trail.

Requirements

• Python 3.9+
• httpx ≥ 0.27

License

MIT