agent-log-ai

An LLM that reads your agent's error and decision logs and drafts the root-cause lessons your keyword heuristics can't.

The third member of the agent-memory family. Where agent-error-log records what BROKE (reactive memory) and agent-decision-log records what was CHOSEN (proactive memory), this repo adds the REASONING layer: it sends the distilled clusters and reversal chains to an LLM-compatible endpoint and drafts the why — the root-cause lesson that keyword frequency and reversal counting can only point at.

Why

The two sibling tools turn an agent's history into mechanical memory: --lessons groups failures by shared keywords, --review counts reversals by topic. Both are deliberately heuristic — deterministic, testable, free. But the reasoning is left to you: why did this cluster keep failing? why did you flip-flop between regex and AST every time the file grew past 200 lines?

That gap is exactly what an LLM is good at. agent-log-ai keeps the heuristics (they select what matters) and hands the explanation to a model — so your memory stops being a list of symptoms and starts being a list of causes.

Quick start

# 1. Point it at a log and draft lessons for the biggest failure cluster
python check_logs_ai.py --lessons --log errors.txt --dry-run   # see the prompt first, costs nothing

# 2. Send it (local-first: no key needed for Ollama/vLLM)
python check_logs_ai.py --lessons --log errors.txt

# 3. Cloud option: any OpenAI-compatible endpoint
OPENAI_API_KEY=sk-... python check_logs_ai.py --lessons --log errors.txt --base-url https://api.openai.com/v1 --model gpt-4o

Commands

Design

• Stdlib only, zero install. The LLM call is urllib.request → POST {base}/chat/completions. No pip packages, same as the siblings.
• Local-first, provider-agnostic. Default --base-url is http://localhost:11434/v1 (Ollama); vLLM and any OpenAI-compatible server work with --base-url. The cloud path (https://api.openai.com/v1) uses OPENAI_API_KEY from the environment — never committed.
• Heuristics point, LLM reasons. The sibling clustering/reversal logic selects what matters; the model explains why. Deterministic pre-processing keeps the prompt small, cheap, and testable.
• Graceful failure. No key and no local server → a clear message, a --dry-run suggestion, and a pointer to the sibling heuristics. Never crashes, never blocks work.
• Cost guard. --max-entries N, line truncation, and a cheap token estimate (chars/4 — approximate by design) that warns before any send.
• Retry manually. Local models load on the first request; if a call times out, run it again (later calls are warm). No automatic backoff in v0.2.0 — --timeout N controls how long a single call waits.

Companion tools

FAQ

Do I need an API key? No — local-first. With Ollama running (ollama serve), the default base-url works with no key. A key is only needed for cloud endpoints.

Does this replace --lessons / --review? No — it uses them. This tool is the reasoning layer on top of the existing heuristics.

Is my log sent somewhere? Only to the endpoint you point at. Local by default; cloud only if you pass --base-url https://... and a key. I copied the tool to a scratch folder — will it read my real logs? No — the --errors / --decisions / --notes-file defaults resolve relative to the script location (HERE), so a scratch copy reads next to itself. Point at your real logs from anywhere with those flags. Does it handle unicode (café, em-dash) on Windows? Yes — stdin is reconfigured to UTF-8 like stdout, so piped unicode prompts are handled without double-encoding.

Development

See CONTRIBUTING.md. Run python _test_logs_ai.py — all tests are offline (the HTTP layer is mocked; the suite runs green with no network and no API key) — (103, 100% pass expected). python check_logs_ai.py must exit 0 on the repo's own log. README test counts are enforced by a drift-guard CI job, so keep them in sync.

Installing with pip (optional)

The single-file adoption story is unchanged - copy check_logs_ai.py into your project and you are done. The tool is also pip-installable with zero runtime dependencies:

pip install agent-log-ai1
log-ai --help

• The package version is derived from the git tag (setuptools-scm), which the release workflow creates from CHANGELOG.md - there is no version to drift.
• Run from the installed package, default paths (decisions.txt, errors.txt, rules.txt, notes.txt) resolve against your current directory; an in-place copy keeps resolving against the file's folder.
• --init works identically from an installed copy (built-in templates).