zeneval
Python library for creating, serving, and evaluating coding benchmarks.
The desktop app (desktop/) is the primary UI; this package exposes the core engine (sample generation, Docker sandboxes, eval harnesses, and the local API in zeneval_api/).
Install
pip install -e ".[api,agents,dev]"
Usage
Load SWE-bench Pro (cached under ~/.cache/zeneval, override with ZENEVAL_CACHE_DIR):
from zeneval.datasets import load
for sample in load("swe_bench_pro"):
print(sample["data"]["instance_id"])
Convert SWE-bench Pro rows into ZenBench generator samples:
from zeneval.pipelines.swe_to_zenbench import SweToZenbenchPipeline
SweToZenbenchPipeline().run(limit=10)
Run generation/eval via the local API (also bundled in the desktop app):
zeneval-api
Package layout
| Module | Purpose |
|---|---|
zeneval.zenbench |
Sample datatypes, generation agent, eval runner |
zeneval_api |
FastAPI server used by the desktop app |
zeneval.sandbox |
Docker sandbox + container registry |
zeneval.harness |
Agent harness integrations (Cursor) |
zeneval.providers |
LLM providers for generation |
zeneval.pipelines |
Benchmark conversion pipelines |
zeneval.datasets |
Optional upstream dataset loaders |
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file zeneval-0.1.1.tar.gz.
File metadata
- Download URL: zeneval-0.1.1.tar.gz
- Upload date:
- Size: 67.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1827213601c9a571987be38ac51813bd950d6be082f4a3a166953fade32ba216
|
|
| MD5 |
ee7435130bda96b55c1e7125f244818f
|
|
| BLAKE2b-256 |
66cdbeb57040dd00464758c3f565c5568c621da55b1afe6bc11dae963555bee8
|
File details
Details for the file zeneval-0.1.1-py3-none-any.whl.
File metadata
- Download URL: zeneval-0.1.1-py3-none-any.whl
- Upload date:
- Size: 91.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
355dd1079db043783ede5b78017a36297fbd0945e272c94edf9966bb89bdf5bb
|
|
| MD5 |
ae1b217557ec6e22cc293ceb64759775
|
|
| BLAKE2b-256 |
8f29c1a46184fc7a905eaee6c5d4396fee24d8a2bc5d1fc30fd9b76953ab7b76
|
