zeno-evals 0.1.2

Visualize OpenAI evals with Zeno

Zeno 🤝 OpenAI Evals

Use Zeno to visualize the results of OpenAI Evals.

Usage

pip install zeno-evals

Run an evaluation following the evals instructions. This will produce a cache file in /tmp/evallogs/.

Pass this file to the zeno-evals command:

zeno-evals /tmp/evallogs/my_eval_cache.jsonl

Example

We include an example looking at the MedMCQA dataset (Thanks to @SinanAkkoyun):

zeno-evals ./example_medicine/example.jsonl --functions_file=./example_medicine/distill.py

Todo

• Support model-graded evaluations
• Support custom evaluation templates (e.g. BLEU for translation)