Zeno 🤝 OpenAI Evals
Use Zeno to visualize the results of OpenAI Evals.
https://user-images.githubusercontent.com/4563691/225655166-9fd82784-cf35-47c1-8306-96178cdad7c1.mov
Example using zeno-evals to explore the results of an OpenAI eval on multiple choice medicine questions (MedMCQA)
Usage
pip install zeno-evals
Run an evaluation following the evals instructions. This will produce a cache file in /tmp/evallogs/.
Pass this file to the zeno-evals command:
zeno-evals /tmp/evallogs/my_eval_cache.jsonl
Example
Single example looking at US tort law questions:
zeno-evals ./examples/example.jsonl
And an example of comparison between two models:
zeno-evals ./examples/crossword-turbo.jsonl --second-results-file ./examples/crossword-turbo-0301.jsonl
And lastly, we can pass additional Zeno functions to provide more context to the results:
pip install wordfreq
zeno-evals ./examples/crossword-turbo.jsonl --second-results-file ./examples/crossword-turbo-0301.jsonl --functions_file ./examples/crossword_fns.py
1 maintainer
Metadata
1 maintainer
Release files for zeno-evals 0.1.10
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| zeno_evals-0.1.10.tar.gz | 4.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| zeno_evals-0.1.10-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 10.0 kB
Release files / zeno_evals-0.1.10.tar.gz
| Download URL | zeno_evals-0.1.10.tar.gz |
|---|---|
| Size | 4.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3ecf12896512ed8ec8bff10cc628c0a840c74b3a7c90c8f068309cf79ac8dbc8
|
|
BLAKE2b-256 checksum How to use checksums |
f03ffd3e7dfa364afea27e0add96cfb03daeb9cee94f0632a80fffb2e5dcfe95
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.8.12 Darwin/22.4.0
|
Release files / zeno_evals-0.1.10-py3-none-any.whl
| Download URL | zeno_evals-0.1.10-py3-none-any.whl |
|---|---|
| Size | 5.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9ee8451d68518fbeba4066f4c48fff2aef5e653299296d67521aa84a03cd677b
|
|
BLAKE2b-256 checksum How to use checksums |
a0f25688d4f1fd25eeaeb850511ac7b01b9fc6c0e9af044648922c56b59fa5d1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.8.12 Darwin/22.4.0
|
