zeno-evals 0.1.1

Visualize OpenAI evals with Zeno

Zeno 🤝 OpenAI Evals

Use Zeno to visualize the results of OpenAI Evals.

Usage

pip install zeno-evals

Run an evaluation following the evals instructions. This will produce a cache file in /tmp/evallogs/.

Pass this file to the zeno-evals command:

zeno-evals /tmp/evallogs/my_eval_cache.jsonl

Example

We include an example looking at the MedMCQA dataset:

zeno-evals example.jsonl

Todo

• Support model-graded evaluations
• Support custom evaluation templates (e.g. BLEU for translation)