openenv-halluguard 1.0.0

Python SDK for HallucinationGuard-Env — evaluate and train LLMs to avoid hallucination using OpenEnv

openenv-halluguard

Python SDK for HallucinationGuard-Env — evaluate and train any LLM to avoid hallucinations using OpenEnv.

Install

pip install openenv-halluguard

Quick Start

from openenv_halluguard import HallucinationGuardEnv

# Define your model function
def my_model(question, context):
    # Call your LLM here — must answer from context only
    return "your answer based on context"

# Evaluate
env = HallucinationGuardEnv()
results = env.evaluate(my_model, episodes=5, model_name="my-model")
env.print_report(results)

What It Does

Connects to the live HallucinationGuard-Env — an OpenEnv RL environment with 100,000+ QA examples across 15 real-world datasets. Your model is scored on:

Component	Weight
Factual correctness	30%
Source grounding	20%
Citation accuracy	15%
Confidence calibration	15%
Semantic consistency	10%
Hallucination penalty	10%

Works With Any LLM

# OpenAI
import openai
client = openai.OpenAI(api_key="sk-...")

def openai_model(question, context):
    r = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(openai_model, episodes=5, model_name="gpt-4")

# Anthropic Claude
import anthropic
client = anthropic.Anthropic(api_key="sk-ant-...")

def claude_model(question, context):
    msg = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=256,
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return msg.content[0].text

env = HallucinationGuardEnv()
results = env.evaluate(claude_model, episodes=5, model_name="claude-haiku")

# Groq (fast inference)
from groq import Groq
client = Groq(api_key="gsk_...")

def groq_model(question, context):
    r = client.chat.completions.create(
        model="llama3-8b-8192",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(groq_model, episodes=5, model_name="llama3-groq")

Optional Dependencies

pip install openenv-halluguard[openai]      # OpenAI
pip install openenv-halluguard[anthropic]   # Anthropic Claude
pip install openenv-halluguard[groq]        # Groq

Links

• 🤗 Live Environment: https://huggingface.co/spaces/SamSankar/hallucination-guard-env
• 📖 API Docs: https://samsankar-hallucination-guard-env.hf.space/docs
• 📦 OpenEnv: https://github.com/meta-pytorch/OpenEnv

License

MIT