nexa-gauge 0.1.2

Agentic LLM evaluation pipeline with cache-aware graph execution

neXa-gauge

Cache-aware LLM evaluation CLI with topology-based execution and branch-level cost estimation.

Install

PyPI (recommended)

pip install nexa-gauge

With Hugging Face adapter support:

pip install "nexa-gauge[huggingface]"

From source (development)

git clone [e-mail]:Sardhendu/nexa-gauge.git
cd nexa-gauge
pip install -e .

Quick Start

# set your provider key
export OPENAI_API_KEY="<your-key>"

# inspect CLI
nexagauge --help

# estimate first
nexagauge estimate grounding --input sample.json --limit 5

# run and write reports
nexagauge run eval --input sample.json --limit 5 --output-dir ./report

CLI Overview

• nexagauge run <target_node> --input <source> [flags]
• nexagauge estimate <target_node> --input <source> [flags]

Most-used flags:

• data: --input, --adapter, --split, --start, --end, --limit
• model routing: --model, --llm-model, --llm-fallback
• cache: --force, --no-cache, --cache-dir
• execution: --max-workers, --max-in-flight, --continue-on-error
• debug: --debug (enables node logs; hides progress bar)
• output (run): --output-dir

Node Topology

Canonical nodes:

• scan
• chunk
• claims
• dedup
• geval_steps
• relevance
• grounding
• redteam
• geval
• reference
• eval
• report

Typical paths:

• grounding: scan -> chunk -> claims -> dedup -> grounding
• relevance: scan -> chunk -> claims -> dedup -> relevance
• geval: scan -> geval_steps -> geval
• eval: full graph execution and aggregation

Configuration

See .env.example for environment settings.

Minimum for LLM-backed runs:

• OPENAI_API_KEY (or alternative provider key)
• LLM_MODEL (default available)

Per-node overrides are supported:

• LLM_{NODE}_MODEL
• LLM_{NODE}_FALLBACK_MODEL
• LLM_{NODE}_TEMPERATURE

For Maintainers

uv sync
make lint
make test
make ci

Releases are automated with release-please:

• merge conventional commits into main (feat:, fix:, deps:) to queue a version bump
• a Release PR is created/updated automatically
• merging that PR creates a v* tag, which triggers .github/workflows/publish.yml

Build distributions:

uv build

Expected artifacts:

• dist/nexa_gauge-<version>-py3-none-any.whl
• dist/nexa_gauge-<version>.tar.gz

Project Standards

• License: MIT
• Security policy: SECURITY.md
• Contributing guide: CONTRIBUTING.md
• Code of conduct: CODE_OF_CONDUCT.md

Documentation

• docs/get-started.md
• docs/architecture.md
• docs/cli-code-flow.md
• docs/PRODUCT_SUMMARY.md