doculayer 0.1.1

  ██████╗  ██████╗  ██████╗██╗   ██╗██╗      █████╗ ██╗   ██╗███████╗██████╗
  ██╔══██╗██╔═══██╗██╔════╝██║   ██║██║     ██╔══██╗╚██╗ ██╔╝██╔════╝██╔══██╗
  ██║  ██║██║   ██║██║     ██║   ██║██║     ███████║ ╚████╔╝ █████╗  ██████╔╝
  ██║  ██║██║   ██║██║     ██║   ██║██║     ██╔══██║  ╚██╔╝  ██╔══╝  ██╔══██╗
  ██████╔╝╚██████╔╝╚██████╗╚██████╔╝███████╗██║  ██║   ██║   ███████╗██║  ██║
  ╚═════╝  ╚═════╝  ╚═════╝ ╚═════╝ ╚══════╝╚═╝  ╚═╝   ╚═╝   ╚══════╝╚═╝  ╚═╝

AI agents hallucinate APIs. Not because they're broken — because their training data is stale.
A function signature that changed six months ago, a parameter that was renamed, a new method that didn't exist at training time — the model confidently fabricates the old behavior.

DocuLayer fixes this. It sits between your agent and the real documentation, fetching live content on demand, returning verbatim text with full source attribution, and never generating a single word.

 Your agent  (Claude, Cursor, Codex, any MCP client…)
     │   "what parameters does httpx.AsyncClient.get() take?"
     ▼
 ┌──────────────────────────────────────────────────────────────┐
 │  DocuLayer                                                   │
 │  ──────────────────────────────────────────────────────────  │
 │  resolve_identifier  →  shortcut table / PyPI / npm         │
 │                                                              │
 │  discover_llms_txt   →  targeted page index (when present)  │
 │       keyword score entries  →  fetch only relevant pages   │
 │                                                              │
 │  DocParser  (HTML → Markdown, heading split)                │
 │  DocSearcher  (BM25 — no ML, no embeddings, no network)     │
 │                                                              │
 │  TTLCache  (in-memory only — zero disk writes)              │
 └──────────────────────────────────────────────────────────────┘
     │   verbatim section + source URL + "fetched 3s ago"
     ▼
 LLM  —  reads real docs, answers correctly

⚡ Quick Install — 30 seconds

Step 1 — Install the package

pip install doculayer==0.1.1

Step 2 — Wire up your IDE (auto-detected, zero prompts)

doculayer setup

That's it. The wizard detects your IDE and writes the MCP config automatically — no editor, no JSON, no manual steps.

Restart your IDE after setup and the four DocuLayer tools appear immediately.

🖥️ IDE-Specific Setup

The doculayer setup command handles all of these automatically.
Manual one-liners are listed below for reference.

Claude Code

pip install doculayer==0.1.1 && claude mcp add doculayer -- doculayer mcp

Restart Claude Code. Tools are live.

Cursor

pip install doculayer==0.1.1 && doculayer setup --ide cursor

Or add manually to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "doculayer": {
      "command": "doculayer",
      "args": ["mcp"]
    }
  }
}

Windsurf

pip install doculayer==0.1.1 && doculayer setup --ide windsurf

Or add manually to ~/.codeium/windsurf/mcp_config.json with the same JSON block above.

VS Code (with MCP support)

pip install doculayer==0.1.1 && doculayer setup --ide vscode

Or add to your VS Code settings.json:

{
  "mcpServers": {
    "doculayer": {
      "command": "doculayer",
      "args": ["mcp"]
    }
  }
}

Zed

pip install doculayer==0.1.1 && doculayer setup --ide zed

Or add to ~/.config/zed/settings.json (macOS/Linux) or %APPDATA%\Zed\settings.json (Windows):

{
  "context_servers": {
    "doculayer": {
      "command": { "path": "doculayer", "args": ["mcp"] }
    }
  }
}

All IDEs at once

doculayer setup --all-ides

🔧 What it does

• MCP server — doculayer_search, doculayer_fetch, doculayer_symbol, doculayer_sources for any MCP client
• Python library — await search("query", "fastapi") / await fetch("httpx", section="AsyncClient") inline in any app
• CLI — doculayer search "streaming responses" --source nextjs
• llms.txt-first — uses the llms.txt index when present to fetch only the 1–3 pages most relevant to the query
• Zero hallucination — every byte returned is verbatim from the fetched URL; attribution header on every response; never generates text
• Zero disk storage — content lives in a process-local TTL cache; no database, no files, no persistence across restarts

🛠️ MCP Tools

Every response includes a citation block:

> **Source**: https://docs.pydantic.dev/latest/concepts/validators/
> **Fetched**: 4s ago

No generated text ever appears in a response.

📦 Python Library

import asyncio
from doculayer import search, fetch

# Search live docs — returns verbatim sections with source attribution
results = asyncio.run(search("dependency injection", "fastapi"))
for r in results:
    print(r.score, r.section.title)
    print(r.section.source_url)     # "https://fastapi.tiangolo.com/tutorial/..."
    print(r.section.content)        # verbatim Markdown from the real docs
    print(r.section.cited_content)  # content with attribution header prepended

# Fetch a specific section
content = asyncio.run(fetch("httpx", section="AsyncClient"))
print(content)
# > **Source**: https://www.python-httpx.org/...
# > **Fetched**: 2s ago
#
# ## AsyncClient
# ...verbatim text...

🔍 Identifier Formats

📚 Packages with llms.txt (fastest access)

These packages publish an llms.txt index that DocuLayer uses to fetch only the pages relevant to your query — instead of the whole docs site.

anthropic · astro · fastapi · httpx · langchain · nextjs · openai · pydantic · react · shadcn · supabase · svelte · tailwindcss · vite · vue

Any package without llms.txt falls back to HTML parsing of the root docs page. Passing a direct URL also works — DocuLayer probes for llms.txt automatically.

⚙️ How it works

1. resolve_identifier("pydantic")
        │
        ▼  shortcut table hit → https://docs.pydantic.dev

2. discover_llms_txt("https://docs.pydantic.dev")
        │
        ▼  fetches /llms.txt → 88 indexed entries

3. _candidate_urls("field validators")
        │
        ▼  keyword-score each entry → top 3 pages
           ["concepts/validators/", "api/validators/", "concepts/models/"]

4. DocFetcher.fetch(url) × 3   (parallel, TTLCache checked first)
        │
        ▼  raw HTML → markdownify → DocParser → list[DocSection]

5. DocSearcher(all_sections).search("field validators", top_k=5)
        │
        ▼  BM25Okapi scores → ranked SearchResult list

6. Return verbatim section content + source URL + fetch timestamp

No embeddings. No vector store. No ML inference. No generated text.

🔒 Storage guarantee

DocuLayer never writes to disk.

• All fetched pages go into a TTLCache[FetchResult] in process memory
• Cache entries expire after DOCULAYER_CACHE_TTL seconds (default: 1 hour)
• On process restart, the cache is empty — nothing persisted
• Safe for privacy-sensitive environments; docs can never go stale past the TTL

🔧 Configuration

All settings are environment variables — no config files, no disk reads:

📊 Compared to RAG

❓ Why not just give the agent a URL?

You could. But:

• The agent still has to know which URL. For a library with 200 pages, it guesses.
• The agent will fetch the whole page and summarize it — that's generation, which means drift.
• DocuLayer uses llms.txt to fetch only the 1–3 pages most likely to answer the query, then returns the relevant section verbatim. The agent reads real documentation, not a paraphrase of it.

🤝 Contributing

git clone https://github.com/inamdarmihir/doculayer
cd doculayer
pip install -e ".[dev]"
pytest

📄 License

Apache 2.0 — see LICENSE.