Python framework for measuring hallucination in AI code review by verifying claims against source.
Project description
magpie
Python framework for measuring hallucination in AI code review by verifying claims against source.
Status: pre-alpha (v0.0.1). Under active development. API is unstable; do not rely on it yet.
Not affiliated with magpie-align/magpie (LLM alignment data synthesis), Blinue/Magpie (Windows upscaler), or any of several "Magpie" companies in adjacent industries.
What it does
LLM code reviewers say things like "function X has bug Y at line N." Most claims are right. Some are confidently wrong (hallucinated). magpie measures how often a given AI reviewer hallucinates on a given codebase, by verifying each claim against actual source.
Install
pip install magpie-eval
(The PyPI distribution is named magpie-eval while the bare magpie namespace is being reclaimed under PEP 541. The Python import name is magpie.)
License
MIT
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file magpie_eval-0.0.1.tar.gz.
File metadata
- Download URL: magpie_eval-0.0.1.tar.gz
- Upload date:
- Size: 2.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.14.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e4446a9c3b7c8bb3d74cb5c984aea38474c4acecb74a74ce37293c7084f7b8e5
|
|
| MD5 |
ce09e9ed7cdc540d2f16aebe8e1a887f
|
|
| BLAKE2b-256 |
0c27839ebd3af21f000ed67f3daab60b5222bca108c781fdff904b37797b1149
|
File details
Details for the file magpie_eval-0.0.1-py3-none-any.whl.
File metadata
- Download URL: magpie_eval-0.0.1-py3-none-any.whl
- Upload date:
- Size: 2.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.14.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
785b011fb36adc39c5c14400b5649e7e4c4195f03ba053fb2e3c8837ce643461
|
|
| MD5 |
7adcd94c04c648c190fe455cc90fbe00
|
|
| BLAKE2b-256 |
f36b42630ad0a6183b5ff12800a31e1d65a5315ee6315c5d25d6dce852ac1d3d
|
