agent-wiki 0.1.0

Toolkit for building agent-maintained Obsidian-style wikis — link management, linting, document conversion, and agent coordination

agent-wiki

Toolkit for building LLM-maintained wikis. Handles the plumbing — link management, linting, file operations, document conversion, and agent coordination — so LLMs focus on content.

Based on the LLM Wiki pattern by Andrej Karpathy: instead of RAG (re-deriving knowledge on every query), the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files. The wiki is a compounding artifact: cross-references are already there, contradictions already flagged, synthesis already current. You curate sources and ask questions; the LLM does the bookkeeping.

Install

pip install agent-wiki

Quick Start

from agent_wiki import WikiRoot

wiki = WikiRoot("/path/to/wiki")

# Health check — find broken links, missing backlinks, orphan pages
issues = wiki.lint()

# Move a page and auto-update every [[link]] across the wiki
wiki.move("topics/Old Name.md", "topics/New Name.md")

# Convert a PDF to structured markdown with images
wiki.convert_pdf("paper.pdf", "processed/paper.md", max_dpi=150)

# Find all pages linking to a topic
wiki.backlinks("Sand Injectites")

# Search for a concept across all pages
wiki.find_references("polygonal fault")

# Wiki statistics
wiki.stats()

CLI

Every operation is also available from the command line — designed for AI agents calling via Bash.

# Lint
agent-wiki --root wiki/ lint
agent-wiki --root wiki/ lint --json          # structured output for agents

# File operations (auto-updates all links)
agent-wiki --root wiki/ move old.md new.md
agent-wiki --root wiki/ merge source.md target.md
agent-wiki --root wiki/ rename "Old Name" "New Name"

# Document conversion
agent-wiki --root wiki/ convert pdf paper.pdf processed/paper.md --max-dpi 150

# Search
agent-wiki --root wiki/ backlinks "Page Name"
agent-wiki --root wiki/ find-references "some concept"
agent-wiki --root wiki/ stats

Use --json on any command for machine-readable output.

Features

Link Management

Obsidian-style [[wiki-links]] with full support for [[target|display text]]. The library builds a link graph, resolves links by filename stem (Obsidian's shortest-unique-path matching), and rewrites links automatically when files move.

from agent_wiki.links import parse_links, find_backlinks, rewrite_links

links = parse_links("See [[Topic A]] and [[Topic B|related topic]]")
# → [WikiLink(target="Topic A", ...), WikiLink(target="Topic B", display="related topic", ...)]

rewrite_links(text, "Old Name", "New Name")
# → updates [[Old Name]] → [[New Name]], preserves [[Old Name|display]] → [[New Name|display]]

Linting

Automated wiki health checks that catch real problems:

Check	Severity	What it detects
Broken links	error	[[Target]] where no file matches
Missing backlinks	warning	Topic lists source but source doesn't link back
Orphan pages	warning	Pages with zero inbound links
Broken breadcrumbs	error	Navigation chain has dead links
Missing frontmatter	error	Required YAML fields missing per page type
Missing sections	warning	Topic without ## Sources, source without ## Topics
Dispute chronology	warning	Disputed claims with dates out of order
Split candidates	info	Pages exceeding 500 lines

issues = wiki.lint()
for issue in issues:
    print(f"[{issue.severity.value}] {issue.file}: {issue.message}")

File Operations

Move, rename, or merge pages — all [[wiki-links]] across the entire wiki are updated automatically.

# Rename a page — finds it by name, renames file, updates all references
wiki.rename("Old Topic Name", "New Topic Name")

# Merge two pages — appends content, redirects all links, deletes source
wiki.merge("sources/duplicate.md", "sources/canonical.md")

Document Conversion

PDF to structured markdown using pymupdf4llm — proper headings, paragraphs, tables, and extracted images. Not flat text.

wiki.convert_pdf(
    "paper.pdf",
    "processed/paper.md",
    max_dpi=150,           # image resolution cap
    extract_images=True,   # images saved to img/ subfolder
)

Stubs for .docx, .pptx, .xlsx conversion are included for future implementation.

Kanban Agent Pipeline

A filesystem-based kanban system for coordinating multiple AI agents. Agents communicate through task cards — lightweight markdown files that move between columns.

from agent_wiki.kanban import create_card, claim, complete, list_cards

# Create a task card (auto-generated by kanban_process)
create_card(
    source_file="raw/paper.pdf",
    processed_file="processed/paper.md",
    kanban_dir="kanban/backlog/",
    agent="reader",
)

# Agent claims work (atomic move — prevents race conditions)
card = claim("kanban/backlog/paper.md", "kanban/processing/")

# Agent finishes — move to next stage
complete("kanban/processing/paper.md", "kanban/review/", agent="writer")

Batch processing with kanban_process:

# Scan for new files, convert, create task cards — one call
cards = wiki.kanban_process(
    input_dir="raw/",
    output_dir="processed/",
    completed_dir="./completed",      # relative to input_dir
    kanban_dir="kanban/backlog/",
)
# → converts new PDFs, moves originals to raw/completed/, creates task cards

Pipeline pattern:

raw/paper.pdf
  → kanban_process() converts + creates card
    → reader agent claims → writes source page → passes to writer
      → writer agent claims → writes topic pages → passes to orchestrator
        → orchestrator reviews → approves to wiki or sends back with actions

No database. The filesystem is the state. Agents coordinate by moving files.

Project Layout

agent-wiki works with any Obsidian-compatible wiki. A typical project:

my-wiki-project/
├── raw/                    # source documents (immutable)
│   └── completed/          # originals moved here after processing
├── wiki/                   # ← this is your WikiRoot
│   ├── processed/          # converted markdown (auto-generated)
│   ├── sources/            # source pages (one per paper)
│   ├── topics/             # topic pages (synthesized knowledge)
│   ├── kanban/             # task cards for agent coordination
│   │   ├── backlog/
│   │   ├── processing/
│   │   ├── review/
│   │   └── done/
│   ├── index.md
│   └── log.md
├── instructions/           # agent prompts and workflow docs
└── agent-wiki.yaml           # optional config

Configuration

Optional agent-wiki.yaml at the project root:

root: wiki/
kanban: wiki/kanban/
processed: wiki/processed/
raw: raw/
completed: raw/completed/

Or configure programmatically:

wiki = WikiRoot("wiki/", kanban_dir="wiki/kanban/", processed_dir="wiki/processed/")

The Pattern

The LLM Wiki pattern has three layers:

1. Raw sources — your curated documents. Immutable. The LLM reads but never modifies.
2. The wiki — LLM-generated markdown. Summaries, topic pages, cross-references. The LLM owns this entirely.
3. The schema — instructions that tell the LLM how the wiki is structured and what workflows to follow.

Three operations:

• Ingest — process a new source into the wiki. Creates a source page, updates topic pages, maintains cross-references.
• Query — answer questions from the wiki. Good answers get filed back as new pages.
• Lint — health-check the wiki. Find broken links, orphan pages, contradictions, missing cross-references.

The human curates sources and asks questions. The LLM does the bookkeeping.

For the full pattern description, see Andrej Karpathy's LLM Wiki gist.

Requirements

• Python 3.12+
• Dependencies: pymupdf, pymupdf4llm, Pillow

License

MIT