11 projects
jevkit-calibrate
Verify TypeSafe Jev's calibration on your own data. Reliability diagrams, ECE, Brier, and confidence thresholds derived from labeled outcomes.
jevkit-bench
Score a labeled TypeSafe Jev suite for accuracy and cost, and compare two runs. Catches regressions aggregate accuracy hides.
jevkit-lint
Static linter for TypeSafe Jev questions. Catches the documented jev-1.13 failure modes before you spend a token. No API key required.
jevkit-drift
Detect when a new TypeSafe Jev model version flips decisions your code depends on. Replays a golden set and diffs the answers.
jevkit-pytest
Record and replay TypeSafe Jev requests in pytest. Cassettes are .jevl files, so a test recording doubles as a drift golden set.
jevkit-core
Shared substrate for jevkit: the .jevl record format, canonical request digests, question normalization, and token budgets for TypeSafe's Jev.
cmd-risk
Heuristic advisory risk classifier for shell commands, with a forkable rule table, for deciding when to prompt a human.
cassette-fn
Record real calls once, replay them deterministically in tests, at the function boundary you choose.
ctx-compact
Trim a plain OpenAI-shaped message array down to a token budget without ever orphaning a tool result.
apply-edit-block
Apply search/replace edit blocks from coding agents to source text, with a fallback ladder for near-exact matches.
prompt-cache-fit
Reorder LLM prompt blocks from least- to most-volatile so provider prefix caches stay hot, and measure whether it actually worked.
