dhrona

Dhrona measures how agents use their tools and teaches better routes. It reads Ramabana and Leela session history, reports tool use and failures, flags poor routes, and turns reviewed sessions into a library of accepted rounds that any Urai chat can replay as a warm start.

The loop

1. Measure — dhrona-tools reports calls, failure rate and shell bypasses per tool and per model; dhrona scores routes in a session.
2. Capture — dhrona-capture turns a finished Ramabana session into an Aidialog review notebook.
3. Review in Leela — open the notebook, remove detours and sensitive content, keep the route future models should imitate.
4. Accept — dhrona-accept --install records the reviewer, the tools used and the model, compiles canonical history, and installs it in the rounds library.
5. Warm start — warm_start(tools, model) replays accepted rounds whose recorded calls still bind to the offered tools’ live signatures, best-ranked and same-model rounds first.

Re-run dhrona-tools after a change to see whether tool use moved.

Install

pip install dhrona

Measure tool use

dhrona-tools measures tool use across the Ramabana and Leela histories: calls, failure rate and denials per tool, shell commands that bypass a dedicated tool, and the same split per model. It is the before/after instrument for tool culling.

dhrona-tools --since 2026-09-19
dhrona-tools --json > baselines/tools.json

assess_turn scores observable tool actions. It does not infer quality from the assistant narrative.

turn = {
    'prompt': 'Use fossick to research https://github.com/AnswerDotAI/llmdojo',
    'activity': [{'tool': 'web_search', 'ok': True, 'args': {'query': 'llmdojo'}}],
}
assessment = assess_turn(turn)
assessment

Findings are route (generic search before FOSSICK), schema (arguments that failed to decode), bypass (a shell command a dedicated tool covers), tool_protocol (command-line usage error), repeat_failure and denial_retry (a refused call repeated instead of a question to the user). Every finding lowers the score by 20 points, including each distinct bypass in a turn, so shelling out for git diff or cat costs points. Run dhrona to assess the default Ramabana history. Pass --session to limit the report to one session.

A refused approval is not a tool failure. Ramabana records it as a failed ask row whose detail starts Denied by human operator and carries the operator’s reason (or the timeout that stood in for one). Dhrona never counts these as schema or repeat_failure; dhrona-tools reports them separately as denials per tool with the top denial_reasons, and assess_history flags denial_retry when the same call is made again in that turn or the next turn of the session without a question to the user in between. The packaged approval-refused seed teaches the alternative: name the refused edit, say nothing changed, and ask once.

Curate a round

Ramabana already records completed turns. Capture reads that archive after the conversation ends. It does not monitor a live process.

ramabana --root /path/to/project
# finish the useful conversation, then quit
dhrona-capture training/github-research.ipynb --session latest
leela training

Open github-research.ipynb in Leela. Remove detours and sensitive content. Keep the route that future models should imitate. Save the notebook, then accept it with a reviewer name.

dhrona-accept training/github-research.ipynb --reviewer Karthik --install

Acceptance updates the notebook metadata with the reviewer, the tools the round calls and the session model, and writes github-research.json as derived canonical history. --install also copies it into the rounds library (~/.config/dhrona/rounds/), where warm_start finds it. An unaccepted notebook cannot start a session.

dhrona-start prints the Ramabana bootstrap and resume commands by default. --launch runs them.

dhrona-start training/github-research.ipynb --root /path/to/project
dhrona-start training/github-research.ipynb --root /path/to/project --launch

Ramabana receives the accepted round as its first bootstrap turn and saves it. Dhrona then resumes that session. Leela can use compiled_history directly when it adds prepared-history support.

Warm start a chat

warm_start returns canonical Urai history from the accepted rounds in the library and the seeds packaged with dhrona. Pass the tools you offer (a list of callables or tool names, or a name → callable dict) and only rounds whose recorded calls bind to those signatures (call_valid) are replayed; a round for a renamed or re-parameterised tool is skipped rather than taught. Rounds sort by meta.rank (lower first), then same-model first, and limit keeps the best few. Pass it to any Urai or Rishi chat through messages=, or call prepare_chat on an empty chat.

Seven seeds ship with dhrona, each one short session on the shalya 0.1.0 / ramabana 0.2.0 tool set:

• git-flow (rank 10) — git_status → git_diff → git_commit(message, paths) → git_divergence → git_remote(op='push'): git through the git tools, and the commit result’s undo token.
• search-choice (rank 20) — search_code for a concept, grep(regex=False) for literal callers, ls for a folder: which search tool fits which question.
• notebook-edit (rank 30) — notebook_cells → view_cell → edit_cell(path, cell_id, edits): notebooks are edited by cell id with native edits.
• approval-refused (rank 40) — view_file → replace_text(path, edits) refused: name the refused edit, say nothing changed, ask once.
• memory-key (rank 50) — remember(key=) twice, then memory_search: a key replaces a note instead of duplicating it.
• delegation (rank 60) — delegate_search(questions=[...]), then view_file on the cited lines: a sub-agent’s report is a hypothesis until confirmed.
• fossick-github — run_shell with fossick read-gh-repo as the first research call for a GitHub repository.

def run_shell(command, cwd='.', timeout=60): ...
history = warm_start([run_shell])
[(message['role'], message.get('name')) for message in history]

A completion receipt includes the package version and round revision. completion_valid rejects receipts from an older round.

Move sessions between hosts

Aidialog notebooks are the interchange format. Ramabana, Claude, and Codex sessions can all be imported for review in Leela.

dhrona-import ramabana training/round.ipynb --session latest
dhrona-import claude training/round.ipynb --session latest --cwd /path/to/project
dhrona-import codex training/round.ipynb --session latest --cwd /path/to/project

The same reviewed notebook can become a Ramabana bootstrap, a resumable Claude session, or Codex-native Responses items.

dhrona-export training/round.ipynb ramabana --output training/round.txt
dhrona-export training/round.ipynb claude --cwd /path/to/project
dhrona-export training/round.ipynb codex --output training/round-items.json

Claude Code can resume the id printed by the Claude export. Codex export does not create a resumable rollout because llmsurgery has no public rollout writer. The exported items remain suitable for inspection, datasets, and a future Codex launcher.

Command line

Develop

uv sync --all-extras --group dev
uv run nbdev-prepare