10 projects
libhallubench
LibHalluBench - evaluate LLM code generation for hallucinated libraries.
dataframebench
DataFrameBench - BigCodeBench problems rewritten to be dataframe-library agnostic.
libreusebench
LibReuseBench - a library-agnostic, test-backed benchmark for measuring library reuse.
thinkpack
A lightweight framework for reasoning-aware training, parsing, and evaluation of explicit reasoning language models.
lure
lure (Library Usage REwards) - reward functions that score LLM code for reusing libraries.
langchoicebench
langchoicebench — evaluate programming language choice in LLMs.
llm-codegen-research
Useful classes and methods for researching code-generation by LLMs.
github-issue-prompter
Use AI to find GitHub issue's that you can work on (even if the issue's appear active)!
jldc
Simplify using JSONLines files alongside dataclasses.
sonora
A WSGI and ASGI compatible grpc-web implementation.
