atlas-eval 0.2.0

Evaluate your AI agent or LLM/SLM on the Bharat AI Index / Atlas Agent Arena.

atlas-eval (Python)

Evaluate your AI agent / LLM on the Bharat AI Index — Atlas Agent Arena.

Requires a running Atlas instance (default http://localhost:3000). Start one with npm run dev in the repo, or docker run -p 3000:3000 ... bharat-ai-index.

Install

pip install atlas-eval           # live on PyPI

Use

from atlas_eval import evaluate_agent, AccuracyEval, PerformanceEval, ReliabilityEval

# whole-agent benchmark (sandboxed tasks, agentic score)
def my_agent(ctx):
    return {"tool": "fs_read", "args": {"path": "invoice_a.txt"}}
report = evaluate_agent(my_agent, base_url="http://localhost:3000")

# code-first evals (Agno-style)
AccuracyEval(agent=lambda q: "New Delhi", input="Capital of India?",
             expected_output="New Delhi").run(print_results=True)
PerformanceEval(func=lambda: my_model("ping"), num_iterations=5).run(print_results=True)
ReliabilityEval(tool_calls=["search"], expected_tool_calls=["search"]).run(print_results=True)

PerformanceEval and ReliabilityEval run fully locally (no server). evaluate_agent and AccuracyEval call the Atlas instance (and a configured judge model for accuracy).

CLI

atlas-eval --version
atlas-eval check --base-url http://localhost:3000   # verify a reachable Atlas instance

What you get back

evaluate_agent returns the full agentic run: agenticScore, successRate, efficiency, toolValidity, safetyPass, and per-task results with trajectories. The eval classes return typed results (AccuracyResult, PerformanceResult, ReliabilityResult).

• Zero runtime dependencies (Python stdlib only). Ships type hints (PEP 561 py.typed).
• Docs & JS equivalents: https://github.com/info-gallary/BharatAIndex/tree/main/sdk

MIT © Atlas AI Labs