eval-adapter

Export rubric-spec rubrics and normalize evaluation result shapes across common eval frameworks.

eval-adapter is for evaluation infrastructure engineers migrating rubric and result contracts between tools. Its differentiator is one inspectable EvalRunResult shape with output_id, criterion_id, raw score, weight, weighted score, runner identity, and source metadata.

What It Actually Adapts

  • run creates deterministic synthetic normalized results for the bundled Inspect AI, LM Eval Harness, OpenAI Evals, PromptFoo, DeepEval, LangSmith, and Phoenix runner modules.
  • LangSmith feedback exports and Phoenix trace exports can be normalized through their Python import helpers.
  • export writes starter rubric files for LM Eval Harness, Inspect AI, OpenAI Evals, or PromptFoo.

The package does not import or execute those frameworks. Generated files are scaffolds that require framework-specific datasets, scorers, providers, and review before production use.

Inspectable Output

eval-adapter run writes canonical JSON to stdout. eval-adapter export writes a target-specific YAML or Python starter file plus the original rubric.json, then reports every written path as JSON.

Runtime Boundary

All operations are local file parsing, deterministic normalization, and file generation. Runtime dependencies are PyYAML and rubric-spec. There are no network requests, model calls, provider credentials, or hosted state.

Install

python -m pip install eval-adapter==0.1.2

For development from a clone:

python -m pip install -e .

Quickstart

From a repository checkout:

eval-adapter run \
  --config examples/unified_config_sample.yaml \
  --runner all \
  > normalized-results.json

Documentation

Release Status

Registry status verified July 13, 2026: version 0.1.2 is published on PyPI and tagged v0.1.2 in the public repository. The project is alpha software. No framework partnership, production-compatibility, or adoption claim is made.

Limits

Normalized synthetic results and generated starter files are not evidence that an external framework ran successfully. This is not a hosted evaluation platform and includes no customer data.

Next Action

Run the sample config with --runner all, verify criterion ids and weights in one normalized result, then export only the intended target and validate that scaffold with the target framework's own tooling.

Project links

Data verified by PyPI on Jul 13, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for auraoneai from gravatar.com auraoneai

Credits

Author: AuraOne

GitHub Statistics

Data verified by PyPI on Jul 13, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.10

Tags

llm-evaluation evals rubric inspect-ai promptfoo lm-eval-harness evaluation-adapter

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

eval_adapter-0.1.2.tar.gz (10.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

eval_adapter-0.1.2-py3-none-any.whl (10.9 kB view details)

Uploaded Python 3

File details

Details for the file eval_adapter-0.1.2.tar.gz.

File metadata

  • Download URL: eval_adapter-0.1.2.tar.gz
  • Upload date:
  • Size: 10.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for eval_adapter-0.1.2.tar.gz
Algorithm Hash digest
SHA256 e7bc48234db7d475ad955000ba58cf4ba7677709196bce00e8e97b66beec9345
MD5 d2f284b738e93dabb994be3ba8b382ff
BLAKE2b-256 2c565745e61f0bab8ec34fd0cd410d5a9e98f724485071d5c0edd3e76ef901e9

See more details on using hashes here.

Provenance

The following attestation bundles were made for eval_adapter-0.1.2.tar.gz:

Publisher: release-python.yml on auraoneai/eval-adapter

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file eval_adapter-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: eval_adapter-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 10.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for eval_adapter-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 564ec8e0e7379cfb481e751b42678f93038bf06a2432542e3a81efbf09d50d66
MD5 14030bdfba3cec15d29e0f61dda2d01e
BLAKE2b-256 6edc34b1c5ea942d3d556e5a7cd4460642f819292fbe7259dad96823bf38076d

See more details on using hashes here.

Provenance

The following attestation bundles were made for eval_adapter-0.1.2-py3-none-any.whl:

Publisher: release-python.yml on auraoneai/eval-adapter

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page