dhrona

Dhrona measures how agents use their tools and teaches better routes. It reads Ramabana and Leela session history, reports tool use and failures, flags poor routes, and turns reviewed sessions into a library of accepted rounds that any Urai chat can replay as a warm start.

The loop

1. Measure — dhrona-tools reports calls, failure rate and shell bypasses per tool and per model; dhrona scores routes in a session.
2. Capture — dhrona-capture turns a finished Ramabana session into an Aidialog review notebook.
3. Review in Leela — open the notebook, remove detours and sensitive content, keep the route future models should imitate.
4. Accept — dhrona-accept --install records the reviewer, the tools used and the model, compiles canonical history, and installs it in the rounds library.
5. Warm start — warm_start(tools, model) replays accepted rounds whose recorded calls still bind to the offered tools’ live signatures, same-model rounds first.

Re-run dhrona-tools after a change to see whether tool use moved.

Install

pip install dhrona

Measure tool use

dhrona-tools measures tool use across the Ramabana and Leela histories: calls and failure rate per tool, shell commands that bypass a dedicated tool, and the same split per model. It is the before/after instrument for tool culling.

dhrona-tools --since 2026-09-19
dhrona-tools --json > baselines/tools.json

assess_turn scores observable tool actions. It does not infer quality from the assistant narrative.

turn = {
    'prompt': 'Use fossick to research https://github.com/AnswerDotAI/llmdojo',
    'activity': [{'tool': 'web_search', 'ok': True, 'args': {'query': 'llmdojo'}}],
}
assessment = assess_turn(turn)
assessment

Findings are route (generic search before FOSSICK), schema (arguments that failed to decode), bypass (a shell command a dedicated tool covers), tool_protocol (command-line usage error) and repeat_failure. Every finding lowers the score by 20 points, including each distinct bypass in a turn, so shelling out for git diff or cat costs points. Run dhrona to assess the default Ramabana history. Pass --session to limit the report to one session.

Curate a round

Ramabana already records completed turns. Capture reads that archive after the conversation ends. It does not monitor a live process.

ramabana --root /path/to/project
# finish the useful conversation, then quit
dhrona-capture training/github-research.ipynb --session latest
leela training

Open github-research.ipynb in Leela. Remove detours and sensitive content. Keep the route that future models should imitate. Save the notebook, then accept it with a reviewer name.

dhrona-accept training/github-research.ipynb --reviewer Karthik --install

Acceptance updates the notebook metadata with the reviewer, the tools the round calls and the session model, and writes github-research.json as derived canonical history. --install also copies it into the rounds library (~/.config/dhrona/rounds/), where warm_start finds it. An unaccepted notebook cannot start a session.

dhrona-start prints the Ramabana bootstrap and resume commands by default. --launch runs them.

dhrona-start training/github-research.ipynb --root /path/to/project
dhrona-start training/github-research.ipynb --root /path/to/project --launch

Ramabana receives the accepted round as its first bootstrap turn and saves it. Dhrona then resumes that session. Leela can use compiled_history directly when it adds prepared-history support.

Warm start a chat

warm_start returns canonical Urai history from the accepted rounds in the library and the seeds packaged with dhrona. Pass the tools you offer (a list of callables or tool names, or a name → callable dict) and only rounds whose recorded calls bind to those signatures (call_valid) are replayed; a round for a renamed or re-parameterised tool is skipped rather than taught. Pass it to any Urai or Rishi chat through messages=, or call prepare_chat on an empty chat.

def run_shell(command, cwd='.', timeout=60): ...
history = warm_start([run_shell])
[(message['role'], message.get('name')) for message in history]

A completion receipt includes the package version and round revision. completion_valid rejects receipts from an older round.

Move sessions between hosts

Aidialog notebooks are the interchange format. Ramabana, Claude, and Codex sessions can all be imported for review in Leela.

dhrona-import ramabana training/round.ipynb --session latest
dhrona-import claude training/round.ipynb --session latest --cwd /path/to/project
dhrona-import codex training/round.ipynb --session latest --cwd /path/to/project

The same reviewed notebook can become a Ramabana bootstrap, a resumable Claude session, or Codex-native Responses items.

dhrona-export training/round.ipynb ramabana --output training/round.txt
dhrona-export training/round.ipynb claude --cwd /path/to/project
dhrona-export training/round.ipynb codex --output training/round-items.json

Claude Code can resume the id printed by the Claude export. Codex export does not create a resumable rollout because llmsurgery has no public rollout writer. The exported items remain suitable for inspection, datasets, and a future Codex launcher.

Command line

Develop

uv sync --all-extras --group dev
uv run nbdev-prepare