repowiki 0.4.2

Live Demo · Quick Start · How It Works · Why RepoWiki? · 中文

Open-source DeepWiki alternative — generate comprehensive wiki documentation for any codebase from your terminal or browser.

The live demo is RepoWiki eating its own dog food: the wiki for this very repo, generated by repowiki scan . --site and served from GitHub Pages.

Why RepoWiki?

Quick Start

pip install repowiki

# set your API key (DeepSeek, OpenAI, Anthropic, etc.)
export DEEPSEEK_API_KEY=<your-api-key>
# or
repowiki config set api_key <your-api-key>

# scan a local project
repowiki scan ./my-project

# scan a GitHub repo
repowiki scan https://github.com/pallets/flask

# scan a private GitHub repo (token never touches logs or errors)
GITHUB_TOKEN=ghp_xxx repowiki scan https://github.com/acme/private-repo

# generate self-contained HTML
repowiki scan ./my-project --format html --open

# start the web interface (wheels from PyPI ship the built UI)
pip install repowiki[web]
repowiki serve ./my-project   # optionally preload a project

RepoWiki respects .gitignore and .repowikiignore during scans. It also skips common local secret files such as .env, .env.local, .npmrc, .pypirc, and SSH private keys by default.

Features

• Structured wiki — project overview, per-module docs, auto-detected architecture with Mermaid diagrams, and a PageRank "start here" reading path.
• Cross-linked pages: a backticked symbol or file path that matches another wiki page becomes a link to it, as a relative .md link in Markdown and as in-page navigation in the HTML export. Fenced code blocks stay untouched, and a name defined on several pages links to the first one.
• Symbol index: a global index page collects every key symbol the analysis documents, grouped by kind and then by module, with each entry linking back to the module page that owns it. A project with no documented symbols skips the page.
• Knowledge cards: a cards page distills every module to one card — purpose, file count, the symbols and concepts it exposes, its internal links, and a jump to the full module page. It is the fastest way to pick a starting point before reading anything longer.
• Incremental re-runs: the output directory keeps a .repowiki-state.json mapping each page to the inputs that generated it, so re-scanning only regenerates pages whose source changed and deletes pages of removed modules. JSON and HTML exports skip the write entirely when nothing changed. Pass --full to force a full rebuild. Unchanged pages also skip the LLM call itself: analysis results sit in a content-keyed SQLite cache (~/.repowiki/cache.db), so a re-scan after a small edit costs no API calls for untouched modules. To auto-refresh on commit, trigger a scan from .git/hooks/post-commit (repowiki scan . --site -o docs/wiki &) or from CI on push — the caches make that cheap, so no watcher daemon is needed.
• Import-aware ranking — resolves Python and JS/TS imports before ranking files, and skips minified/generated bundles so they don't burn LLM context.
• Symbol skeletons for oversized files — a Python module too big for the per-file context budget used to contribute only its first 4,096 characters. Now the analyzer sees an ast-derived skeleton instead: every top-level class and function with its signature and docstring, so a 2,000-line module is read by structure, not by head.
• Honest coverage reporting — when the scan can't take the whole repo, it says so: the overview page and the CLI both flag partial coverage (files kept vs. candidates, oversized and excluded paths), so a wiki never quietly claims to be complete.
• Three output formats — a Markdown directory to commit, structured JSON, or a self-contained HTML file to share (diagrams included).
• Static site publishing: repowiki scan . --site drops a docsify loader (index.html + .nojekyll) into the Markdown export, so the output directory can go straight onto GitHub Pages.
• Web viewer + terminal chat: a three-column browser UI, or repowiki chat . for grounded Q&A in the terminal. Chat is multi-turn: the conversation so far goes into each prompt, so follow-up questions work in both the web UI and the CLI. Every answer carries its sources at line precision: the CLI prints a file:start-end footer under each reply, and the web UI links each reference into a file viewer that opens the exact line range (/project/<id>/file/<path>#L120-L140-style links survive refresh and sharing). The built-in TF-IDF index (no embeddings service) persists across runs and reuses per-file chunks, so a second session on an unchanged repo starts warm and an edit to a few files rebuilds only their chunks instead of the whole index.
• CLI-first — no Docker, no database server, no browser required.

repowiki scan .                    # generate wiki
repowiki scan . --full             # rebuild every page, ignoring incremental state
repowiki scan . -f html --open     # open in browser
repowiki scan . -l zh              # Chinese output
repowiki chat .                    # multi-turn Q&A about the code, remembers the session
repowiki map .                     # ranked repo map, zero LLM calls
repowiki map . --format json       # prompt-ready ranked list for agents
repowiki scan . --site             # markdown export plus a GitHub Pages-ready loader

Languages & Models

Detects Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin, C/C++, C#, Ruby, PHP, Swift, and 30+ more. Any of litellm's 100+ providers works — pick one with an alias or pass it directly:

repowiki config set model deepseek   # deepseek / claude / gpt / gemini / qwen / kimi / glm ...
repowiki scan . -m gpt               # or pass a model directly

Configuration

RepoWiki looks for config in this order:

1. CLI flags (-m, -l, -o)
2. Environment variables (REPOWIKI_MODEL, REPOWIKI_API_KEY)
3. Config file (~/.repowiki/config.json)
4. Provider-specific env vars (DEEPSEEK_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY)

For private GitHub repositories, set GITHUB_TOKEN (or GH_TOKEN): the clone goes out authenticated and the token is used only inside the git invocation, never written to logs or error output.

Project Structure

RepoWiki/
├── src/repowiki/
│   ├── cli.py              # Click CLI with scan/serve/chat/config commands
│   ├── config.py           # Configuration management
│   ├── core/
│   │   ├── scanner.py      # File scanning with language detection
│   │   ├── analyzer.py     # Multi-step LLM analysis pipeline
│   │   ├── graph.py        # Dependency graph + PageRank
│   │   ├── wiki_builder.py # Wiki page assembly
│   │   ├── rag.py          # TF-IDF retrieval for Q&A
│   │   ├── cache.py        # SQLite caching
│   │   └── state.py        # Incremental regeneration state
│   ├── llm/
│   │   ├── client.py       # litellm async wrapper
│   │   └── prompts.py      # Structured prompt templates
│   ├── ingest/
│   │   ├── local.py        # Local directory ingestion
│   │   └── github.py       # Git clone with caching
│   ├── export/
│   │   ├── markdown.py     # Markdown directory export
│   │   ├── json_export.py  # JSON export
│   │   └── html.py         # Self-contained HTML export
│   └── server/             # FastAPI web backend
├── frontend/               # React + Vite + TailwindCSS
├── pyproject.toml
└── LICENSE

How It Works

1. Scan — Walk the directory tree, filter out binaries, generated bundles, and oversized files, detect languages and entry points
2. Graph — Resolve imports across 6 languages, including Python package-relative and JavaScript/TypeScript relative modules, then run PageRank to rank file importance
3. Analyze — Send file tree + key files to LLM in 4 structured passes (overview, modules, architecture, reading guide)
4. Cache — Store results in SQLite keyed by content hash, skip unchanged files on re-scan
5. Export — Assemble wiki pages with Mermaid diagrams and source links, output in chosen format

Development

git clone https://github.com/he-yufeng/RepoWiki.git
cd RepoWiki

# backend
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,web]"

# frontend
cd frontend && npm install && npm run dev

# run backend
repowiki serve --port 8000

# tests + retrieval eval
pytest tests/
python evals/run_eval.py

The retrieval eval (evals/) runs fixture repos through the real ingest and index path and checks that questions with known source files still retrieve them. It runs blocking inside pytest and as a soft (non-blocking) gate in CI; run python evals/run_eval.py --update-baseline after an intentional retrieval change.

Roadmap

Generation, the web interface, and the diagrams work, pages link to each other, re-runs only regenerate the pages whose source changed, and scan --site exports a GitHub Pages-ready site. The next step is richer diagrams:

• More diagram types — a call graph and a data-flow view alongside the dependency graph, since the analysis already walks imports and could surface more.

Related Projects

If RepoWiki helped you find your way around a codebase, a few other things I've built:

• CoreCoder — want to understand how a coding agent really works? Read the whole ~1k-line engine end to end, not a black box.
• FindJobs-Agent — stop sifting job boards by hand: it ranks postings against your resume and runs mock interviews.
• ContractGuard — catch the risky clauses before you sign: it reads contracts and flags the dangerous bits.
• GitSense — want to contribute to open source? It finds issues worth your time and gauges whether your PR will get merged.
• CodeABC — understand any codebase even if you don't code, built for non-programmers.

License

MIT