zeno-evals 0.1.3

Visualize OpenAI evals with Zeno

Zeno 🤝 OpenAI Evals

Use Zeno to visualize the results of OpenAI Evals.

https://user-images.githubusercontent.com/4563691/225655166-9fd82784-cf35-47c1-8306-96178cdad7c1.mov

Example using zeno-evals to explore the results of an OpenAI eval on multiple choice medicine questions (MedMCQA)

Usage

pip install zeno-evals

Run an evaluation following the evals instructions. This will produce a cache file in /tmp/evallogs/.

Pass this file to the zeno-evals command:

zeno-evals /tmp/evallogs/my_eval_cache.jsonl

Example

We include an example looking at the MedMCQA dataset (Thanks to @SinanAkkoyun):

zeno-evals ./example_medicine/example.jsonl --functions_file=./example_medicine/distill.py

Todo

• Support model-graded evaluations
• Support custom evaluation templates (e.g. BLEU for translation)