ParetoOps

Cost per successful task for LLM applications. ParetoOps reads your eval results (Promptfoo, LangSmith, Braintrust, DeepEval, CSV or JSON), finds the Pareto frontier of cost against accuracy, recommends the cheapest configuration that meets your accuracy bar, and gates CI on it.

pip install pareto-ops
pareto-ops analyze results.json
pareto-ops gate results.json --config my-config --min-acc 0.9

The pareto-ops command is the same native CLI that npm install -g pareto-ops installs.

Python API

from pareto_ops import ParetoOptimizer, EvalTrial, ParetoAnalysisOptions, load_trials

trials = load_trials("promptfoo_results.json")  # or build EvalTrial objects yourself
result = ParetoOptimizer(trials).compute(options=ParetoAnalysisOptions(min_success_rate=0.85))

print(f"Sweet spot: {result.sweet_spot.config_name}")
for rec in result.recommendations:
    print(f"[{rec.type}] {rec.description}")

Free and Pro

Free (no key, no sign-up): analyze, gate on accuracy and cost thresholds, every importer, the pricing catalog and this Python API. In the Python API, the 3D frontier (enable_3d_optimization) and confidence-bound ranking (use_statistical_confidence) are Pro: pass license_key=... to compute() or set PARETO_LICENSE_KEY. Pro also adds the regression gate against your last good run, the git-branch baseline store, early stopping (watch) and the CI gate with PR comments (pareto-ops ci). Pro is coming soon: join the interest list.

License keys are verified offline. ParetoOps sends no telemetry and never uploads your data.

Platforms

Prebuilt wheels for CPython 3.9 and newer on Linux (x86_64, arm64; glibc and musl), macOS (Intel and Apple silicon) and Windows (x64). Other platforms can use the standalone binary via npm.

License

Proprietary. Use is governed by the ParetoOps End-User License Agreement, included in the wheel as LICENSE. Third-party notices are in THIRD_PARTY_NOTICES.md.