culprit

git bisect thinking for prompts: compare what changed in behavior, not only in text.

Not to be confused with the unrelated culprit package on PyPI or the Culprit Rust CLI — see How this compares.

Features

Semantic diff: an LLM summary backed by per-case eval evidence.
Behavioral bisect: binary-search prompt history to locate the first regression.
Semantic merge (experimental): conservatively combine compatible branch edits and verify the result before staging it.
Git-native architecture: prompt history stays in ordinary Git.
Provider adapters for OpenAI, Anthropic, and arbitrary OpenAI-compatible endpoints.
Content-addressed evaluation cache to avoid repeat API spend.
culprit check warnings for common secret and PII patterns.

How this compares

Among these similarly named tools, culprit-ai is differentiated by verifying prompt diff, bisect, and automated merge decisions against real eval runs rather than relying on text similarity or an LLM's judgment alone.

Tool	Primary focus	Verification model
`culprit` on PyPI	Local, read-only root-cause and change-risk analysis for PRs, branches, fixes, and traces	Deterministic static analysis, with optional Git bisect confirmation when a repro command is available
Culprit Rust CLI	A local-first, guided Git bisect workflow with optional AI explanation	Runs the supplied test command at candidate commits and restores the checkout afterward
`culprit-ai` (this project)	Prompt-behavior diff, behavioral bisect, and experimental semantic merge in one Git-native toolkit	Runs the configured eval set; merge additionally rejects a candidate that loses behavior shared by both parents

Other prompt tools cover adjacent workflows and can complement culprit-ai:

Tool	Best fit	How this project's focus differs
Promptfoo	Broad prompt/model evaluation and red teaming	`culprit-ai` centers comparisons on Git revisions and commit history.
Confident AI / DeepEval	Evaluation frameworks and hosted workflows	`culprit-ai` is a local Git wrapper with no required hosted backend.
promptvc	Prompt versioning workflows	`culprit-ai` leaves version storage to Git and adds eval-backed behavioral operations.

Installation

pip install culprit-ai

Quickstart

cp examples/quickstart/culprit.yaml .
cp -r examples/quickstart/evals .
export OPENAI_API_KEY="..."
culprit diff HEAD~1 HEAD --eval-set evals/summarizer.yaml

Contributing

Semantic diff, behavioral bisect, and experimental semantic merge improvements are open for contribution. See CONTRIBUTING.md, including adapter guidance and suggested first issues.

Released under the MIT License.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

culprit_ai-0.1.5.tar.gz (235.6 kB view details)

Uploaded Sep 9, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

culprit_ai-0.1.5-py3-none-any.whl (28.3 kB view details)

Uploaded Sep 9, 2026 Python 3

File details

Details for the file culprit_ai-0.1.5.tar.gz.

File metadata

Download URL: culprit_ai-0.1.5.tar.gz
Upload date: Sep 9, 2026
Size: 235.6 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for culprit_ai-0.1.5.tar.gz
Algorithm	Hash digest
SHA256	`1e7d695f48a0527befae92d8382b27dc24e00f40d5a81f7cbb4b4f4151b1b25d`
MD5	`d73d191f85e43a9d01d6bd7480942b3a`
BLAKE2b-256	`6bab0408e5cecfdffec43b4ac3690f63516e41f0350453237ea71840bb39f2ab`

See more details on using hashes here.

Provenance

The following attestation bundles were made for culprit_ai-0.1.5.tar.gz:

Publisher: release.yml on abi6374/culprit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: culprit_ai-0.1.5.tar.gz
- Subject digest: 1e7d695f48a0527befae92d8382b27dc24e00f40d5a81f7cbb4b4f4151b1b25d
- Sigstore transparency entry: 2766074656
- Sigstore integration time: Sep 9, 2026
Source repository:
- Permalink: abi6374/culprit@a4da62907a5ffd604b56bf1295656a69e83f458e
- Branch / Tag: refs/tags/v0.1.5
- Owner: https://github.com/abi6374
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@a4da62907a5ffd604b56bf1295656a69e83f458e
- Trigger Event: push