kompend
kompend refuses to pick a side in the LLM-wiki space. Existing tools either demand a frontier-model subscription and lock you into their stack, or insist everything run on an underpowered local model while lecturing you about privacy. kompend is provider-agnostic, adapts to whatever LLM you have — from a local 8B model to a frontier API — and uses structure-preserving parsing so technical content (tables, equations, figures) survives extraction intact. Read the full motivation →
OKF v0.2 compliant — produces Open Knowledge Format bundles by default. ACP agent — implements the Agent Client Protocol for credential-free LLM injection via
kompend agent. PydanticAI-native — every LLM call routes through typed agents with failover chains; models.dev catalog resolves any provider.
How it works
kompend runs a 5-tier pipeline — each tier independently replaceable, with a
clear Base interface:
| Tier | What it does | Needs LLM? |
|---|---|---|
| T1 Preprocessing | Parse PDF, DOCX, PPTX, EPUB, XLSX, HTML, URLs into a uniform ParsedDocument preserving structure (headings, tables, figures, equations, code). |
❌ |
| T2 Chunking | Split into a ChunkTree sized to the model's context window (auto-detected). Atomic elements never split; text runs via recursive paragraph→line→sentence→token boundaries. |
✅ |
| T3 Resolution | Extract entities, resolve cross-references, acronym/definition consistency. Optional VLM enricher for figure captioning. | ✅ |
| T4 Compilation | Build an interlinked CompiledWiki graph. Two backends: TwoPhaseCompiler (default, extract→generate) and WiCERCompiler (iterative compile→evaluate→refine). |
✅ |
| T5 Output | Write OKF bundles (default), Markdown, JSON, or Typst. Optionally index for semantic search and synthesize an overview. | ❌ |
kompend ingest stops at T1 — no LLM, no cost. kompend compile runs T2–T5.
Quick start
uv tool install kompend
# 1. Initialize a project
kompend init ./my-wiki && cd ./my-wiki
# 2. Ingest sources (T1 only — no LLM needed)
kompend ingest # everything in sources/
kompend ingest paper.pdf notes.docx *.xlsx # explicit files
kompend ingest --url https://arxiv.org/pdf/2605.07068v1
# 3. Compile the wiki (T2–T5 — needs an LLM)
kompend compile --set compile.llm.default_model=ollama/qwen3
# 4. Query it
kompend query "How does attention scale with sequence length?"
Debugging tip:
kompend parse paper.pdf --elementsinspects one file's extracted structure before ingesting the full corpus. If parsing looks wrong, fix it first — the pipeline won't recover from bad extraction.
Commands
| Command | Purpose |
|---|---|
kompend init [DIR] |
Scaffold a wiki project (--bare, --force, --git) |
kompend ingest FILES... |
Parse sources into .kompend/cache/ (T0–1, no LLM) |
kompend compile FILES... |
Full pipeline → interlinked wiki (T2–5, needs LLM) |
kompend query "..." |
Ask a natural-language question against the wiki |
kompend organize |
LLM-categorize sources/ into a hierarchy and move files |
kompend parse FILES... |
Debug: parse documents and print structure (--elements) |
kompend doctor |
Validate dirs, parsers, LLM/VLM connectivity |
kompend status |
Show source/article/word counts and last compile time |
kompend config get/set/list/path |
Inspect and edit configuration (5-layer precedence) |
kompend lint run |
Lint wiki: 8 structural + 4 LLM rules (--fix, --llm) |
kompend review list/approve/reject |
Manage the review queue for compiled pages |
kompend capture "..." / learn / distill |
Capture notes → synthesize into wiki pages |
kompend write "TITLE" |
One-shot wiki page from a title + optional sources |
kompend provenance [SLUG] |
Trace a page to its source files |
kompend memory add/search/synthesize |
Manage query memory and context synthesis |
kompend serve |
Start the web UI (--host, --port, --open) |
kompend mcp |
Start the MCP server (stdio, for AI clients) |
kompend agent |
Start the ACP agent (editor integration) |
Full option reference: kompend <command> --help
Key subsystems
| Subsystem | Description |
|---|---|
| RAG / Search | HybridRetriever fuses FTS + vector + token-overlap + raw sources via reciprocal rank fusion. Embeddings via litellm: prefix or sentence-transformers. |
| Memory lifecycle | Confidence decay, tier promotion (working → episodic → semantic → procedural), supersession detection. |
| Review queue | Route compiled pages through approve/reject/lock before publishing. |
| MCP server | Stdio server exposing parse_document, compile_wiki, wiki_capture, wiki_learn to Claude Code, Cursor, etc. |
| ACP agent | Agent Client Protocol server: editors inject LLM credentials via providers/set — no API keys in your config. |
| Configuration | 5-layer precedence: CLI > env (KOMP_*) > project (.kompend/config.toml, walks up) > global (~/.config/kompend/) > defaults. |
LLM configuration
kompend uses provider/model strings resolved by build_model():
- models.dev catalog — custom providers with known API URLs (
claudinio/,opencode/,deepseek/) construct OpenAI-compatible models. - pydantic-ai native — standard providers (
ollama/,openai/,anthropic/,google/) viainfer_model().
API keys come from the env var each provider advertises. No api_key in config:
[compile.llm]
default_model = "ollama/qwen3"
fallback_chain = ["openai/gpt-4o"] # tried when the primary fails
reasoning_effort = "none" # tame over-thinking local models
kompend auto-detects each model's context window and sizes every call to fit. Recommended: a model with ≥ 32K context (16K is the functional floor; the generation stage reserves 16K output tokens). See the LLM configuration guide for the full breakdown.
See the LLM configuration guide for the full reference.
Documentation
See
docs/index.mdfor the complete documentation index — architecture deep-dives, tier references, WiCER loop, CLI guides, integrations.
Development
git clone <repo-url> && cd kompend
uv sync # install deps
uv run kompend --help # verify CLI
uv run pytest tests/ # run tests
mise test # run all quality checks
Installing kompend as a dependency
Install from PyPI:
uv add kompend
# or
pip install kompend
The published package is kompend (latest release tracks the latest YYYY.M.N tag; current is 2026.10.4). Optional extras for parser backends and LLM providers are declared in the repository's pyproject.toml and resolve as uv add "kompend[<extras>]" or pip install "kompend[<extras>]".
License
MIT. See LICENSE.
Acknowledgments
- Karpathy's compiled LLM-wiki Gist — the core concept
- Rohit G00's LLM Wiki v2 Gist — memory lifecycle, knowledge graph, hybrid search
- WiCER paper (Huerta, Zinnia Tech) — iterative compilation loop
- Docling (IBM) — structure-preserving parsing
- Chonkie — multi-strategy chunking
- pydantic-ai — typed agent framework
- models.dev — provider/model catalog
1 maintainer
Metadata
1 maintainer
Release files for kompend 2026.10.24
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| kompend-2026.10.24.tar.gz | 14.0 MB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| kompend-2026.10.24-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 14.9 MB
Release files / kompend-2026.10.24.tar.gz
| Download URL | kompend-2026.10.24.tar.gz |
|---|---|
| Size | 14.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1eefb48bfd7af4046b628b498b6620e01d7bccf7116d1b13f0448d726b5f251a
|
|
BLAKE2b-256 checksum How to use checksums |
23d7854a0dac7edb688eea9ad02d90bce122ef1f0275a22017b22cffa34bc842
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logRelease files / kompend-2026.10.24-py3-none-any.whl
| Download URL | kompend-2026.10.24-py3-none-any.whl |
|---|---|
| Size | 854.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b99fa7e9cb05b76bf03ec4beb96ff1e68ef7c2a647f163c01fc847a440dbed0d
|
|
BLAKE2b-256 checksum How to use checksums |
7ae271d4148eeab400fb7fd901de3e6df1722273e0d721cf91670c29fb3692e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logSecurity for kompend 2026.10.24
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for kompend-2026.10.24.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
