Debug code generation models
Project description
CodeGaze [Beta]: A library for evaluating and debugging code generation models
Still in development.
Code gaze implements a set of evaluation metrics and visualization tools for debugging code generation models.
CodeGaze is build around a set of abstractions that allow for the evaluation of code generation models.
- Dataset: An example code generation dataset e.g. humaneval.
- Experiment: A set of parameters that define the evaluation of a code generation model. Each experiment specifies things like the dataset, model properties (temperature, n_completions), and some metric properties.
- Model: A code generation model that can be evaluated. This is either an OpenAI model or a HuggingFace model.
The basic starting point is to run an experiment on a dataset with a list of models.
Installation
pip install codegaze
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
codegaze-0.0.21a0.tar.gz
(1.0 MB
view hashes)
Built Distribution
codegaze-0.0.21a0-py3-none-any.whl
(693.6 kB
view hashes)
Close
Hashes for codegaze-0.0.21a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fee7be386d56fd3ea332ea92c9471afabb715968ee73efa15d5c9ac333aa9152 |
|
MD5 | 9b5b28cd47395723317ae121b6aef3ec |
|
BLAKE2b-256 | acc23652e6a572717d7afe220862ce32a6403a42e3bbdd01a72bf5306c1efabc |