writing-context-rtfm 0.6.1

MCP extension for writing-context retrieval relying on RTFM

Surgical Context for Writing Agents

Stop giving your AI agent the entire manuscript to write one section. Give it the exact paragraphs, constraints, and dependencies it needs to succeed. No token bloat. No hallucinations.

Lightweight · Task-Focused · Extension · MIT

---

Your writing agent is drowning in tokens.

You ask Claude or Cursor to "Write the methodology section." To give it context, you feed it your 50-page manuscript, your related works, and your notes. The agent gets overwhelmed by the global narrative, loses track of the specific hyper-parameters you wanted to include, and writes a generic, repetitive summary that reads like a high-school essay.

The bottleneck isn't the model's writing ability — it's the noise.

writing-context-rtfm fixes the noise. It is a lightweight MCP extension built on top of rtfm-ai. Instead of letting the agent grep freely, it acts as a gatekeeper. It takes the agent's task, queries the underlying RTFM index, aggressively filters out background noise, and packs only the essential and supporting source chunks into a tight, highly-focused prompt.

writing-context-rtfm pack \
  --task "Write the methodology section detailing dataset and quantization" \
  --target sections/methodology.tex \
  --budget 4000

3 seconds later, the agent receives a compact context pack containing exactly the paragraphs and key terms needed, alongside stylistic constraints for the target section. The agent writes perfectly.

Token budgets respected. Constraints enforced. Progressive disclosure over context dumps.

---

Installation & Onboarding

writing-context-rtfm is published on PyPI and runs as a Model Context Protocol (MCP) server.

1. Install writing-context-rtfm

You can install the package globally or in your virtual environment:

# Using uv (recommended)
uv tool install writing-context-rtfm

# Using pipx
pipx install writing-context-rtfm

2. Install the RTFM CLI (Retrieval Engine)

Since writing-context-rtfm queries and relies on the rtfm-ai database, you must install the rtfm-ai command-line tool to initialize and synchronize your manuscript's retrieval index:

# Using uv (recommended)
uv tool install "rtfm-ai[embeddings]"

# Using pipx
pipx install "rtfm-ai[embeddings]"

(Note: If you are setting up inside a local virtual environment, running uv pip install "writing-context-rtfm[tiktoken]" will automatically pull in rtfm-ai[embeddings] as a library dependency, but installing it globally ensures the rtfm binary is available on your PATH).

3. Install Zotero MCP (Semantic Literature Grounding)

writing-context-rtfm uses zotero-mcp to ground your agent in your local literature library. It supports dynamic semantic search over your PDFs and metadata. You must install zotero-mcp and have Zotero Desktop running for literature grounding to work:

# Install zotero-mcp globally using uv
uv tool install zotero-mcp

(Make sure Zotero Desktop is open while the agent is running so it can connect to the local SQLite database).

---

4. Quick Project Onboarding

To integrate the server into your manuscript repository, run the following commands:

Step A: Initialize configuration and editor rules

writing-context-rtfm init

This command non-destructively:

• Creates a self-documenting .writing-context/config.yaml file template showing how to tune token budgets and role weights.
• Appends the cache database path to your .gitignore.
• Updates your local .mcp.json to register the MCP server automatically.
• Adds Agent Rules of Thumb blocks into CLAUDE.md, AGENTS.md, and GEMINI.md to guide AI agents on retrieving context first and respecting LaTeX boundaries.

Step B: Auto-scaffold your section cards

writing-context-rtfm cards build

This scans your workspace for LaTeX files, parses \input structures, maps section dependencies, and uses model-assisted inference to automatically scaffold purposes, key terms, and constraints. It outputs the generated structure to .writing-context/cards.generated.yaml.

(Note: If you do not have or want to use an OpenAI API key, writing-context-rtfm supports a model fallback chain for card scaffolding: OpenAI API -> Hugging Face Serverless Inference API (requires HF_TOKEN, defaults to Qwen/Qwen2.5-Coder-7B-Instruct) -> Local Ollama server (running at http://localhost:11434, defaults to qwen2.5-coder or phi3) -> Deterministic Offline Scan fallback).

Step C: Initialize, Sync and Setup Embeddings

Initialize the RTFM index inside your repository and generate the semantic search embeddings:

# 1. Initialize RTFM configuration
rtfm init

# 2. Run the initial sync to build the index database
rtfm sync

(Note: writing-context-rtfm init only configures the writing-context settings, cards, and agent rules; it does not automatically initialize or sync the underlying RTFM database. This setup assumes you already have at least part of the .tex files in your repository—if starting from an empty repository or using Overleaf, ensure your files are placed locally first).

Baseline Model Embeddings

• Default Local Model: By default, RTFM automatically generates embeddings for all document chunks. It uses a fast, lightweight multilingual model (sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) which runs completely locally on your CPU/GPU and downloads automatically from Hugging Face on the first sync. No external API keys are required.
• Customizing Models: If you want to use a larger or different model, you can run the embedding step explicitly:

  # Options: fast (default), balanced (BAAI/bge-base-en-v1.5), quality (mixedbread-ai/mxbai-embed-large-v1)
  rtfm embed --embed-model balanced
  

OpenAI Semantic Search (Optional Extension)

If you prefer to leverage OpenAI embeddings for semantic expansion instead of running local transformer models, writing-context-rtfm includes a built-in provider that seamlessly overlays OpenAI vectors onto RTFM's index.

1. Securely save your API key to the local cache: writing-context-rtfm auth openai_semantic "sk-..."
2. Enable it in your .writing-context/config.yaml:

   providers:
     openai_semantic:
       enabled: true
       model: "text-embedding-3-small"
       auto_sync: false # Set to true to embed all files automatically during `rtfm sync`
   

Note: This architecture uses an ultra-fast numpy in-memory comparison, requiring zero C++ SQLite VSS extensions, ensuring maximum compatibility across all operating systems in your laboratory.

Card Scaffolding Generator Configuration (Optional)

To customize the model or API endpoint used during cards build / cards infer (for example, to use a local Ollama instance or the Hugging Face Inference API instead of OpenAI), configure the generator block in your .writing-context/config.yaml:

generator:
  # The model to use (e.g., gpt-4o-mini, Qwen/Qwen2.5-Coder-7B-Instruct, phi3)
  model: "Qwen/Qwen2.5-Coder-7B-Instruct"
  # The API endpoint base URL (e.g. https://api-inference.huggingface.co/v1, http://localhost:11434/v1)
  api_base: "https://api-inference.huggingface.co/v1"
  # The API key/token (optional; falls back to environment variables or local auth cache)
  # api_key: "your-token"

• Hugging Face Inference API: Query using a free Hugging Face token. You can save your token locally using: writing-context-rtfm auth huggingface "hf_...".
• Local Ollama Server: Start Ollama and run ollama pull phi3 or ollama pull qwen2.5-coder. Point api_base to http://localhost:11434/v1 and set model to your pulled model name. No API key is required.

---

MCP Server Integration

1. Claude Desktop

Add this to your claude_desktop_config.json (on macOS: ~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "writing-context-rtfm": {
      "command": "writing-context-rtfm",
      "args": [
        "serve"
      ]
    }
  }
}

2. Cursor IDE

1. Open Cursor Settings (Cmd + ,).
2. Navigate to Features > MCP and click + Add New MCP Server.
3. Name: writing-context-rtfm
4. Type: command
5. Command: writing-context-rtfm serve

3. VS Code Extensions (Cline, Roo Code)

Update your MCP settings file (e.g., cline_mcp_settings.json):

{
  "mcpServers": {
    "writing-context-rtfm": {
      "command": "writing-context-rtfm",
      "args": [
        "serve"
      ]
    }
  }
}

4. Claude Code (Anthropic CLI Agent)

# Global configuration
claude mcp add --scope user --transport stdio writing-context-rtfm -- writing-context-rtfm serve

# Repository-local configuration
claude mcp add --scope local --transport stdio writing-context-rtfm -- writing-context-rtfm serve

---

The Core Philosophy: RTFM Retrieves, We Pack

Tool	Role	Action	Output
rtfm-ai	The Retrieval Layer	Indexes everything, runs FTS/Semantic search, returns raw hits.	25 raw chunks
writing-context-rtfm	The Curation Layer	Filters noise, applies constraints, ranks by structural priority.	4 essential chunks

We do not replace or fork RTFM. We wrap it. RTFM is built to fetch memory. writing-context-rtfm is built to decide what is enough memory to write a specific section.

---

Features

1. Agent Self-Correction Loop

If the requested token budget is too small to fit the necessary target context, the packer returns a "status": "degraded" and appends a warning to the warnings list recommending a specific minimum budget:

• Writing Packs (pack): To resolve this, call the tool with a larger token_budget of at least X.
• Proofreading Packs (proofread-pack): To resolve this, call the tool with a larger max_tokens value of at least X.

AI client agents are instructed via the auto-injected guidelines to parse this warning, extract the recommended value X, and automatically retry the call with the new budget to get the missing context.

2. LaTeX Safety Checks

The extension automatically parses the target text and detects LaTeX math environments, macro calls, and cross-references (e.g., \ref{...}, \begin{equation} ... \end{equation}). It issues safety warnings to the AI writing agent containing the exact code patterns that must not be deleted or broken during edits, maintaining compilation safety.

3. SQLite Local Caching

To optimize token and response latency, generated context packs are hashed and cached locally in .writing-context/context_cache.sqlite. Cache keys are dynamically invalidated whenever the project configuration, section cards, or underlying RTFM indexes are updated.

4. Semantic Zotero Grounding

The extension dynamically routes literature queries to zotero-mcp.

• Semantic Routing: High-level section intents (e.g., "Discuss how smart cities use IoT") are routed to ChromaDB-powered semantic searches.
• Proofread Protection: If you run a proofread-pack, the engine intelligently skips open-ended searches to prevent "context contamination" (hallucinating new ideas into your polish phase), only resolving the explicit \cite{} keys already present in the draft.

---

The Split-Cards Pattern (Overrides & Generated)

To give writing agents context and rules, we define manuscript metadata. Rather than forcing you to maintain a single massive YAML configuration manually, writing-context-rtfm splits section cards into two layers:

1. cards.generated.yaml (Machine-Written): Generated automatically by writing-context-rtfm cards build. The tool scans your LaTeX file tree, maps structural hierarchies, and uses local/remote models to extract default purposes, key terms, facts, and constraints. Do not modify this file.
2. cards.overrides.yaml (Human-Controlled): The user control panel. Create or edit this file to override generated settings or declare global document parameters (like style guidelines, project-wide glossary, or specific section rules).

At runtime, the extension automatically overlays cards.overrides.yaml on top of the generated metadata, compiling them into a single unified context card database.

Override File Example (cards.overrides.yaml)

version: 2

# Project-wide global context rules
document:
  title: "A New Approach to Manuscript Curation"
  thesis: "Surgical context selection using a gatekeeping protocol reduces LLM token overhead."
  writing_style:
    tone: "Academic, precise, third-person"
    avoid_words: ["groundbreaking", "revolutionary", "game-changing"]
  terminology:
    Context Pack: "A compact JSON structure containing prioritized source spans and constraints."

# Override specific sections generated by the tool
sections:
  section_methodology:
    title: "Proposed Methodology"
    purpose: "Detail the system architecture and context selection algorithms."
    depends_on:
      - section_introduction
    must_preserve:
      - "Token budget formula is B_usable = B_total * (1 - margin)"
    avoid: ["premature results discussion"]
    constraints:
      - "Write equations using LaTeX align environments"

Configurable Zotero Grounding

If Zotero is enabled in your .writing-context/config.yaml, the provider routes keyword/semantic queries to Zotero. To prevent out-of-domain paper recommendations from cluttering your prompt, configure Zotero's semantic filtering under extra in config.yaml:

providers:
  zotero:
    enabled: true
    mcp_server:
      command: zotero-mcp
    extra:
      # Optimal threshold: -0.4. Discards negative similarity noise (like off-topic papers) 
      # while preserving relevant priority queueing and network scheduling matches.
      similarity_threshold: -0.4
      include_abstract: false  # Set to true to include full abstracts in Zotero spans

How the Agent Applies Cards

When the agent requests context to write section_methodology, the system:

1. Expands the query: Uses the target section title, override keywords, and task text to search.
2. Enforces Exclusions (Avoids): Instantly discards any retrieved text matching your defined avoid list.
3. Injects Rules & Terminology: Directs the agent to respect constraints (e.g. must_preserve) and glossary terms during writing.

[!IMPORTANT] Modular documents are required. This extension's noise-reduction algorithms heavily rely on the path defined in your section cards to perform "Target Boosts" and semantic scoping. If your entire manuscript is just a single monolithic main.tex or main.md file, the packer won't be able to distinguish the target section from background noise. Keep your writing modular (e.g., sections/01_intro.tex, sections/02_methodology.tex) for optimal results.

---

CLI Reference

# Initialize project config, gitignore, and editor rules
writing-context-rtfm init

# Build section cards (scan, infer, and update in sequence)
writing-context-rtfm cards build

# Deterministically scan the manuscript structure
writing-context-rtfm cards scan

# Interactively review generated card candidates
writing-context-rtfm cards review

# Initialize the local SQLite cache database (.writing-context/context_cache.sqlite)
writing-context-rtfm init-db

# Run diagnostics health checks on databases and configuration files
writing-context-rtfm doctor

# Sync the underlying RTFM index
writing-context-rtfm sync

# Generate a context pack directly in the terminal
writing-context-rtfm pack \
  --task "Update the introduction" \
  --target sections/introduction.tex \
  --budget 4000

# Generate a proofreading context pack
writing-context-rtfm proofread-pack sections/abstract.tex --line-start 1 --line-end 10 --max-tokens 3000

# Inspect configured rules and details for a specific section card
writing-context-rtfm inspect-target --target section_abstract

# Look up a term in the document glossary config
writing-context-rtfm get-term "Context Pack"

# Show the LaTeX reference graph and section dependencies
writing-context-rtfm show-graph

# Clear the cached context packs
writing-context-rtfm cache clear

# Show cache database size and run statistics
writing-context-rtfm cache stats

# Start MCP Server
writing-context-rtfm serve

---

Where this fits

┌─────────────────────────────────┐
│       AI Agent / LLM Client     │  ← Execution (Cursor, Claude)
├─────────────────────────────────┤
│     writing-context-rtfm        │  ← Curation (Packs, Filters, Rules)
├─────────────────────────────────┤
│           rtfm-ai               │  ← Retrieval (Index, FTS, Semantic)
└─────────────────────────────────┘

Without the context packer, your agent retrieves 50 documents and hopes for the best. With it, the agent receives a surgically precise, prioritized briefing.

License

MIT License — use it, fork it, extend it.