keeploop 0.2.0

English | Tiếng Việt | 简体中文

Your AI proposes the change. A number decides if it stays.

Quick start · Examples · How it works · FAQ · Full guide

_{Recorded from the real output of the offline mock demo (colours added). The full transcript is in docs/DEMO-OUTPUT.txt.}

AI agents are good at sounding sure. keeploop makes them prove it.

• You say what "better" means: faster tests, a higher benchmark, a cleaner report. One number.
• Your AI tries one change at a time, and a second, fresh AI checks it.
• Only a change that beats the number is kept. Everything else is undone, and the reason is written down so it is not tried again.

Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI and other AIs you already use. keeploop itself is not an AI: it is a small Python tool that measures and refuses, with no dependencies, no telemetry and no API keys.

Only have a chat app (ChatGPT, Claude, Gemini, no code)? Use keeploop-lite: one file you paste into the chat, the same keep-or-discard loop for a post, a CV or a plan (Tiếng Việt, 简体中文, example). Your AI can run code? Use the full version below.

Quick start

(a) Hand it to your AI. If it can run commands, paste this and fill in the last line:

Clone https://github.com/johnha912/keeploop next to my project (not inside it),
run `python prepare.py` in the clone, then read its AGENTS.md and act as the CEO
it describes. What I want better: <for example "make my test suite faster">

You never type kl and there are no flags to learn. The first time, your AI asks in plain words what "better" means and how to measure it, then proposes an evaluate.py (a small script that prints that number) for you to approve before anything changes. In Claude Code, /keeploop 3 inside the clone runs three cycles.

(b) Try it offline. Needs Python 3.10+ and git; no AI, no API key, a few seconds.

git clone https://github.com/johnha912/keeploop.git && cd keeploop
python prepare.py
python run.py --mock --iterations 5

cycle 3: discard  metric=0.942  best=0.992000  tokens=2820
cycle 4: crash  metric=None  best=0.992000  tokens=3510  lesson=L-001
run complete: 3 keeps, 1 discards, 1 crashes, 1 lessons, 17259 tokens. Best metric: 1.000000

The mock is a scripted stand-in for a real AI. It shows how the loop works, not how good a real model is. Every keep is a real commit: git -C .keeploop/workspace log --oneline.

(c) Install it as a command. kl without a clone, from PyPI:

pipx install keeploop      # or: pip install keeploop (in a venv of its own)
mkdir my-loop && cd my-loop
kl init                    # copies AGENTS.md, program.md, team.yaml, packs/, .claude/ here
kl begin                   # the demo baseline

kl init puts the files you edit in your own folder and never overwrites one. Then open the folder in your AI as in (a), and follow AGENTS.md to point it at your repo. kl run is the headless mode. Keep the clone if you want to read or change keeploop itself.

Examples

Set up any example as a sandbox with kl init --example <name> <new short folder> (installed kl) or python tools/make_sandbox.py examples/<name> <new short folder> (clone) (details).

Why keeploop

An agent saying "done" is not a measurement.

Also in the guide: lessons so a mistake is not repeated, a trend scout, and one AGENTS.md that Codex, Cursor, Copilot and others read.

How it works

Built on Andrej Karpathy's autoresearch idea: a fixed metric, a keep-or-discard rule, and git as the memory of what worked.

flowchart LR
    A[Analyst<br/>writes one card] --> E[Engineer<br/>makes the change]
    E --> G{Code gates<br/>fence · protected<br/>secrets · size}
    G -- pass --> V[evaluate.py<br/>one number]
    V --> U[Auditor<br/>fresh eyes,<br/>real diff]
    U --> C{beats best<br/>+ noise?}
    C -- yes --> K[git commit]
    C -- no --> D[git reset]
    G -- fail --> L[(lesson)]
    D -. why .-> L
    L -. read next time .-> A

An Analyst writes one card, an Engineer makes exactly that change, code gates and evaluate.py check it, and a fresh Auditor reads the diff. Only kl keep commits. The full cycle, the critic round, lessons, the trend scout and speed tuning are in the guide.

Your own work: packs

A pack gives the team the craft rules and review checklist for one field.

New field? python kl.py pack new NAME scaffolds one. Setup for your own repo, team.yaml, vendors and the headless overnight mode (python run.py) are in docs/GUIDE.md. Why the experimental ones are experimental: guide.

Safety

• keeploop never executes text from a model or a web page. Only the Engineer gets edit rights; changed files are scanned for credentials, and token-like environment variables are removed before any vendor CLI or evaluation starts.
• While code you did not write runs, every file in the project folder outside the workspace is content-hashed. Any change stops the run.
• The fence detects changes; it is not a sandbox. Run real vendors in a container or VM if that matters to you.

Details: docs/GUIDE.md and SECURITY.md.

FAQ

More questions: docs/GUIDE.md.

Docs

Credits

keeploop stands on autoresearch by Andrej Karpathy: a fixed evaluation the agent cannot touch, an append-only results.tsv, a keep/discard rule and git as memory. keeploop turns it into an org chart any AI can run, and adds roles, cross-vendor audit, code gates, lessons and a trend scout. No code from autoresearch is copied. If you like this repo, star his first.

The banner is set in Google Sans Flex.

If keeploop stopped a bad change from landing, a ⭐ helps other people find it. Bugs and ideas: open an issue.

License

MIT.