Debug code generation models
Project description
CodeGaze [Beta]: A library for evaluating and debugging code generation models
Still in development.
Code gaze implements a set of evaluation metrics and visualization tools for debugging code generation models.
CodeGaze is build around a set of abstractions that allow for the evaluation of code generation models.
- Dataset: An example code generation dataset e.g. humaneval.
- Experiment: A set of parameters that define the evaluation of a code generation model. Each experiment specifies things like the dataset, model properties (temperature, n_completions), and some metric properties.
- Model: A code generation model that can be evaluated. This is either an OpenAI model or a HuggingFace model.
The basic starting point is to run an experiment on a dataset with a list of models.
Installation
pip install codegaze
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
codegaze-0.0.22a0.tar.gz
(1.0 MB
view hashes)
Built Distribution
codegaze-0.0.22a0-py3-none-any.whl
(693.7 kB
view hashes)
Close
Hashes for codegaze-0.0.22a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38c1932d562feae98fb5c1ca018adaf0f6a99e2bb483e2df11b2fab76d67ff21 |
|
MD5 | db6027affd29aad70b98cc2895671a12 |
|
BLAKE2b-256 | f7c1e7928f5961c9321861208774ee1a64ce0e5e0959d72d0df020b99444ac92 |