openenv-halluguard 2.0.0

Python SDK for HallucinationGuard-Env — evaluate and train any LLM to avoid hallucinations using OpenEnv. Research-grade grader: ROUGE + BERTScore + AlignScore + NLI-DeBERTa-v3-large. 1M+ training examples across 38 datasets.

openenv-halluguard

Python SDK for HallucinationGuard-Env — evaluate and train any LLM to avoid hallucinations using the OpenEnv framework.

---

What is this?

HallucinationGuard-Env is a reinforcement learning environment that trains AI models to stop making things up. It provides a reward signal based on 9 research-grade components including ROUGE, BERTScore, AlignScore, and NLI-DeBERTa-v3-large — built on 1M+ real-world QA examples across 38 datasets.

This package is the Python SDK that lets you evaluate any LLM against the environment in 3 lines of code.

---

Install

pip install openenv-halluguard

Optional extras for specific model providers:

pip install openenv-halluguard[openai]     # + openai
pip install openenv-halluguard[anthropic]  # + anthropic
pip install openenv-halluguard[groq]       # + groq (free tier)

---

Quick Start

from openenv_halluguard import HallucinationGuardEnv

# Define your model — must answer from the provided context only
def my_model(question, context):
    # call any LLM here
    return "your answer based on context"

# Evaluate
env = HallucinationGuardEnv()
results = env.evaluate(my_model, episodes=5, model_name="my-model")
env.print_report(results)

---

Works With Any LLM

OpenAI

import openai
from openenv_halluguard import HallucinationGuardEnv

client = openai.OpenAI(api_key="sk-...")

def gpt4(question, context):
    r = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Answer ONLY from the context provided."},
            {"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"}
        ]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(gpt4, episodes=10, model_name="gpt-4o")
env.print_report(results)

Anthropic Claude

import anthropic
from openenv_halluguard import HallucinationGuardEnv

client = anthropic.Anthropic(api_key="sk-ant-...")

def claude(question, context):
    msg = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=256,
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return msg.content[0].text

env = HallucinationGuardEnv()
results = env.evaluate(claude, episodes=10, model_name="claude-3-5-sonnet")
env.print_report(results)

Groq (free tier — Llama, Mixtral)

from groq import Groq
from openenv_halluguard import HallucinationGuardEnv

client = Groq(api_key="YOUR_GROQ_KEY")

def llama(question, context):
    r = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQ: {question}\n\nAnswer from context only."}],
        max_tokens=200
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(llama, episodes=10, model_name="llama-3.1-8b")
env.print_report(results)

Ollama (local, fully offline)

import requests
from openenv_halluguard import HallucinationGuardEnv

def ollama(question, context):
    r = requests.post(
        "http://localhost:11434/api/generate",
        json={"model": "llama3", "stream": False,
              "prompt": f"Context: {context}\n\nQ: {question}\n\nAnswer from context only."}
    )
    return r.json()["response"]

env = HallucinationGuardEnv()
results = env.evaluate(ollama, episodes=5, model_name="llama3-local")
env.print_report(results)

---

GRPO Training (5 lines)

from trl import GRPOConfig, GRPOTrainer
from openenv_halluguard import HallucinationGuardEnv

env = HallucinationGuardEnv(verbose=False)

def reward_fn(prompts, completions, **kwargs):
    return [env.step(c).get("reward", 0.0) for c in completions]

trainer = GRPOTrainer(
    model="Qwen/Qwen3-1.7B",
    reward_funcs=reward_fn,
    args=GRPOConfig(output_dir="./halluguard-grpo", num_train_epochs=1),
)
trainer.train()

---

Reward System (v2.0 — Research-Grade)

9 components, each contributing to the final reward in [0.0, 1.0]:

Component	Weight	Reference
Factual Correctness	30%	Semantic similarity + entity overlap
Source Grounding	25%	Word coverage vs context
Citation Accuracy	10%	Fuzzy match of source_quote in document
Confidence Calibration	8%	|confidence − correctness|
Semantic Consistency	7%	NLI-DeBERTa-v3-large entailment
Hallucination Penalty	5%	Multi-type fabrication detection
ROUGE-L	5%	Lin (2004)
BERTScore	5%	Zhang et al. (2020)
AlignScore	5%	Zha et al., ACL 2023

---

Datasets

1M+ examples cached permanently across 38 real-world QA datasets including SQuAD, TriviaQA, HotpotQA, HaluEval, TruthfulQA, MedQA, AQUA-RAT, HellaSwag, AdversarialQA, and more. No downloads required at runtime — instant boot.

Full dataset list: HuggingFace Space

---

HTTP API

The environment is also available as a live REST API:

import requests

BASE = "https://samsankar-hallucination-guard-env.hf.space"

obs    = requests.post(f"{BASE}/reset").json()
result = requests.post(f"{BASE}/step", json={"answer": "your answer"}).json()

print(result["reward"])           # 0.0 – 1.0
print(result["is_hallucination"]) # True / False
print(result["feedback"])         # Detailed explanation

---

Links

🌐 Site	https://ss-360.github.io/Openenv-halluguard/
🤗 HF Space	https://huggingface.co/spaces/SamSankar/hallucination-guard-env
📖 API Docs	https://samsankar-hallucination-guard-env.hf.space/docs
🏆 Leaderboard	https://samsankar-hallucination-guard-env.hf.space/leaderboard
💾 Dataset Cache	https://huggingface.co/datasets/SamSankar/hallucination-guard-cache
🐛 Issues	https://github.com/SS-360/Openenv-halluguard/issues

---

Build for to Train AI Models to Stop Hallucinating

MIT License