buildlog 0.10.4

Engineering notebook for AI-assisted development

buildlog

A measurable learning loop for AI-assisted work

Track what works. Prove it. Drop what doesn't.

RE: The art. Yes, it's AI-generated. Yes, that's hypocritical for a project about rigor over vibes. Looking for an actual artist to pay for a real logo. If you know someone good, open an issue or DM me. Budget exists.

Read the full documentation

---

The Problem

Most AI agents do not learn. They execute without retaining context. You can bolt on memory stores and tool routers, but if the system cannot demonstrably improve its decision-making over time, you have a persistent memory store, not a learning system.

Every AI-assisted work session produces a trajectory: goals, decisions, tool uses, corrections, outcomes. Almost all of this is discarded. The next session starts from scratch with the same blind spots.

buildlog exists to close that gap. It captures structured trajectories from real work, extracts decision patterns, and uses statistical methods to select which patterns to surface in future sessions, then measures whether that selection actually reduced mistakes.

buildlog measures whether the system actually got better, and proves it.

How It Works

1. Capture structured work trajectories

Each session is a dated entry documenting what you did, what went wrong, and what you learned. Each session is a structured record of decisions and outcomes, not a chat transcript.

buildlog init          # scaffold a project
buildlog new my-feature   # start a session
# ... work ...
buildlog commit -m "feat: add auth"

2. Extract decision patterns as seeds

The seed engine watches your development patterns and extracts seeds: atomic observations about what works. A seed might be "always define interfaces before implementations" or "mock at the boundary, not the implementation." Each seed carries a category, a confidence score, and source provenance.

Extraction runs through a pipeline: sources -> extractors -> categorizers -> generators. Extractors range from regex-based (fast, cheap, brittle) to LLM-backed (accurate, expensive). The pipeline deduplicates semantically using embeddings.

3. Select which patterns to surface using Thompson Sampling

Seeds compete for inclusion in your agent's instruction set. The system treats each seed as an arm in a contextual bandit and uses Thompson Sampling to balance exploration (trying under-tested rules) against exploitation (surfacing rules with strong track records).

Each seed maintains a Beta posterior updated by observed outcomes. Over time, the system converges on the rules that actually reduce mistakes in your specific codebase and workflow, not rules that sound good in the abstract.

4. Render to every agent format

Selected rules are written into the instruction files your agents actually read:

• CLAUDE.md (Claude Code)
• .cursorrules (Cursor)
• .github/copilot-instructions.md (GitHub Copilot)
• Windsurf, Continue.dev, generic settings.json

The same knowledge base renders to every agent format.

buildlog skills   # render current policy to agent files

5. Close the loop with experiments

Track whether the selected rules are working. Run experiments, measure Repeated Mistake Rate (RMR) across sessions, and get statistical evidence, not feelings, about what improved.

buildlog experiment start
# ... work across sessions ...
buildlog experiment end
buildlog experiment report

What Else Is In the Box

• Review gauntlet: automated quality gate with curated reviewer personas. Runs on commits (via Claude Code hooks or CI) and files GitHub issues for findings, categorized by severity.
• LLM-backed extraction: when regex isn't enough, the seed engine can use OpenAI, Anthropic, or Ollama to extract patterns from code and logs. Metered backend tracks token usage and cost.
• MCP server: buildlog exposes itself as an MCP server so agents can query seeds, skills, and build history programmatically during sessions.
• npm wrapper: npx @peleke.s/buildlog for JS/TS projects. Thin shim that finds and invokes the Python CLI.

Current Limits

This is v0.8, not the end state.

• Extraction quality is uneven. Regex extractors miss nuance; LLM extractors are accurate but expensive. The middle ground is still being found.
• Feedback signals are coarse. Repeated Mistake Rate works but requires manual tagging. Richer automatic signals (test outcomes, review results, revision distance) are on the roadmap.
• Credit assignment is limited. When multiple rules are active, the system doesn't yet isolate which one was responsible for an outcome.
• Single-agent only. Multi-agent coordination (shared learning across agents) is designed but not implemented.
• Long-horizon learning is not modeled. The bandit operates per-session. Longer arcs of competence building need richer policy models.

The roadmap: contextual bandits (now) -> richer policy models -> longer-horizon RL -> multi-agent coordination. Each step builds on the same foundation: measuring whether rule changes actually reduce mistakes.

Installation

Always-On Mode (recommended)

We run buildlog as an ambient data capture layer across all projects. One command, works everywhere:

pipx install buildlog         # or: uv tool install buildlog
buildlog init-mcp --global    # registers MCP + writes instructions to ~/.claude/CLAUDE.md

That's it. Claude Code now has all 29 buildlog tools and knows how to use them in every project you open. No per-project setup needed.

The --global flag:

• Registers the MCP server in ~/.claude/settings.json
• Creates ~/.claude/CLAUDE.md with usage instructions so Claude proactively uses buildlog
• Works immediately in any repo, even without a local buildlog/ directory

This is how we use buildlog ourselves: always on, capturing structured trajectories from every session, feeding downstream systems that generate engineering logs, courses, and content.

Per-project setup

If you prefer explicit per-project control:

pip install buildlog          # MCP server included by default
buildlog init --defaults      # scaffold buildlog/, register MCP, update CLAUDE.md

This creates a buildlog/ directory with templates and configures Claude Code for that specific project.

For JS/TS projects

npx @peleke.s/buildlog init

Verify installation

buildlog mcp-test          # verify all 29 tools are registered
buildlog overview          # check project state (works without init in global mode)

Quick Start

buildlog init --defaults      # scaffold + MCP + CLAUDE.md
buildlog new my-feature       # start a session
# ... work ...
buildlog commit -m "feat: add auth"
buildlog experiment start
# ... work across sessions ...
buildlog experiment end
buildlog experiment report

Documentation

Section	Description
Installation	Setup, extras, and initialization
Quick Start	Full pipeline walkthrough
Core Concepts	The problem, the claim, and the metric
CLI Reference	Every command documented
MCP Integration	Claude Code setup and available tools
Experiments	Running and measuring experiments
Review Gauntlet	Reviewer personas and the gauntlet loop
Multi-Agent Setup	Render rules to any AI coding agent
Theory	The math behind Thompson Sampling
Philosophy	Principles and honest limitations

Contributing

git clone https://github.com/Peleke/buildlog-template
cd buildlog-template
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
pytest

We're especially interested in better context representations, credit assignment approaches, statistical methodology improvements, and real-world experiment results (positive or negative).

License

MIT License. See LICENSE