atlas-eval (Python)

Is your model ready for India? Evaluate any model on the Bharat AI Index.

Requires a running Atlas instance (default http://localhost:3000). Start one with npm run dev in the repo, or docker run -p 3000:3000 ... bharat-ai-index.

Install

pip install atlas-eval           # live on PyPI

Use

from atlas_eval import evaluate_model

# point at ANY model — OpenAI-compatible endpoint or local Ollama:
report = evaluate_model("gpt-4o-mini", api_key="sk-...")
# report = evaluate_model("my-finetune", base_url="http://localhost:8000/v1")
# report = evaluate_model("gemma2:2b", kind="ollama")

r = report["readiness"]
print(r["verdict"], "| grade", r["sovereigntyGrade"])
print("Indic", r["indicScore"], "vs Global", r["globalScore"], "| gap", r["indicGap"])
print("weakest axes:", r["weakest"])   # what to fix before deploying for India

CLI

atlas-eval --version
atlas-eval check --base-url http://localhost:3000   # verify a reachable Atlas instance

What you get back

evaluate_model returns the full report:

• run — overall ± stderr, per-axis categories, and the Indic-vs-Global locales split.
• sovereignty — overall score, AAA→C grade, and the four sovereignty dimensions.
• readiness — a plain-English verdict, indicScore / globalScore / indicGap, and the weakest / strongest axes.
• certificateId — a deterministic, recomputable id for the rating.

Zero runtime dependencies (Python stdlib only). Ships type hints (PEP 561 py.typed). Docs: https://github.com/info-gallary/BharatAIndex/tree/main/sdk

MIT © Atlas AI Labs