1. The Problem

Most "AI + data" tools are fetchers. Give them a URL, they hand back a string, and then they forget everything. That string has:

• ❌ no date — you can't tell if it's current or three years stale
• ❌ no provenance — you can't open the exact spot it came from
• ❌ no memory — the next question starts from zero
• ❌ no dedup or conflict detection — ten copies of the same claim look like ten facts
• ❌ no sense of absence — the model can't tell you what it doesn't have, so it fills the gap by hallucinating

flowchart LR
    U["URL / PDF / video"] --> F["🪣 Fetcher"]
    F --> S["raw string<br/>no date · no source · no memory"]
    S --> L["🤖 LLM"]
    L --> A["answer you cannot verify"]
    style F fill:#3a1a1a,stroke:#a33,color:#fff
    style A fill:#3a1a1a,stroke:#a33,color:#fff

When the answer is wrong, you have no way to check why.

2. The Solution

AnchorX is the layer above the fetchers. Every piece of knowledge it acquires is normalized into a single schema and stored in a local, embedded corpus — carrying a resolvable anchor back to a human-openable source, a publication date, a quality score, and freshness metadata.

flowchart LR
    U["URL / PDF / video / image"] --> A["⚓ AnchorX"]
    A --> D[("Document<br/>+ anchors<br/>+ provenance<br/>+ quality")]
    D --> C[("Local corpus<br/>SQLite + LanceDB")]
    C --> L["🤖 LLM (Claude)"]
    L --> R["answer + [20:47] · p.14 · ¶14<br/>citations you can click open"]
    style A fill:#0d2818,stroke:#2ea44f,color:#fff
    style R fill:#0d2818,stroke:#2ea44f,color:#fff

Success criterion: not "the AI stops being wrong." It's "when the AI is wrong, one click shows you exactly why."

Architecture at a glance — read it bottom-up

The durable value lives at L0/L1; the adapters at L2 are replaceable parts.

┌─────────────────────────────────────────────────────────────────────┐
│  L4  INTERFACE      MCP server  ·  CLI  ·  Python SDK                │
│      ask · search · ingest · research · coverage · fetch · clear     │
└────────────────────────────────┬────────────────────────────────────┘
┌────────────────────────────────▼────────────────────────────────────┐
│  L3  ORCHESTRATION  Budget Governor · Job Queue · Research · Sync    │
└────────────────────────────────┬────────────────────────────────────┘
        ┌────────────────────────┴───────────────────────┐
┌───────▼──────────────────────────┐   ┌─────────────────▼────────────┐
│  L2  ACQUISITION (adapters)       │   │  L2' RETRIEVAL               │
│  youtube · web_deep · web_search  │   │  pre-filter → BM25 + dense   │
│  documents · images               │   │  → RRF → cross-encoder rerank│
└───────────────┬───────────────────┘   └─────────────────┬────────────┘
┌───────────────▼─────────────────────────────────────────▼────────────┐
│  L1  NORMALIZATION   RawPayload → Document → Chunks → Vectors         │
│      provenance stamping · anchor assignment · quality scoring        │
└────────────────────────────────┬────────────────────────────────────┘
┌────────────────────────────────▼────────────────────────────────────┐
│  L0  CORPUS   SQLite (metadata · FTS5 · graph) · LanceDB (vectors)   │
│      blob cache · dedup · freshness · coverage · grounding           │
└─────────────────────────────────────────────────────────────────────┘

3. Usage

Install in Claude Code (30 seconds)

claude mcp add anchor -- uvx anchorx

No clone, no virtualenv, no Ollama. On first launch AnchorX auto-installs its browser, downloads a tiny CPU embedding model, creates its data directory, and serves 8 tools to Claude:

anchor_ingest · anchor_ask · anchor_search · anchor_research · anchor_coverage · anchor_fetch · anchor_refresh · anchor_clear

Then just talk to Claude:

"Ingest this PDF and these two articles into Anchor, then answer my question with citations." "What's my coverage on 'RAG'? Any gaps?" "Using only that video, what are the first words spoken?"

Or use the CLI

pip install anchorx

anchor ingest https://youtu.be/VIDEO_ID  "D:/reports/invoice.pdf"  https://example.com/post
anchor ask "what is the total on the invoice"        # grounded, cited, offline
anchor ask "how does chunking work" --url youtu.be   # scope to one source
anchor research "latest techniques for scraping SPAs"# discover → acquire → answer
anchor coverage "retrieval augmented generation"     # what do I have, what's missing
anchor clear --all                                   # wipe the corpus

Or the Python SDK

from anchor_ai.adapters.web_deep import acquire, render_page
from anchor_ai.corpus import DocumentStore, coverage

doc = acquire("https://example.com/post", render=render_page)   # L0→L2 ladder
print(doc.title, doc.quality_score, [s.heading for s in doc.sections])

report = coverage(store.conn, "chunking")
print(report.gaps)   # ["No source newer than 2025-12 …", "62% from one domain …"]

4. Why AnchorX

Four principles drive every decision in the design — and each one is a capability the fetcher-based ecosystem doesn't have.

AnchorX vs. a plain fetcher / vanilla RAG:

5. Technical Details

5.1 Ingestion — from a URL to a grounded, chunked Document

flowchart LR
    U["URL / file"] --> R{Router}
    R -->|youtube| Y[YouTube adapter]
    R -->|.pdf/.docx/.csv| DOC[Documents adapter]
    R -->|image| IMG[Images adapter]
    R -->|web| W[web_deep ladder]
    Y & DOC & IMG & W --> N["Normalize → Document<br/>provenance + anchors"]
    N --> C["Structure-aware<br/>chunking"]
    C --> E["Embed (model2vec, CPU)"]
    E --> S[("SQLite + LanceDB<br/>+ blob cache")]

5.2 Query — filter first, then fuse, then rerank, then ground

flowchart LR
    Q["Question"] --> F["Metadata<br/>pre-filter"]
    F --> B["BM25 (FTS5)"]
    F --> D["Dense (LanceDB)"]
    B --> X["RRF fusion<br/>k = 60"]
    D --> X
    X --> RR["Cross-encoder<br/>rerank → top 8"]
    RR --> A["Answer + citations<br/>+ grounding report"]

5.3 research() — the multi-round acquisition loop (capped at depth 3)

flowchart TD
    A[Question] --> B[Discover candidates]
    B --> C["Score: authority × recency × relevance / cost"]
    C --> D{Within budget?}
    D -->|no| G[Trim to top-N] --> E
    D -->|yes| E[Acquire → normalize → persist]
    E --> H[Retrieve + rerank]
    H --> I[Coverage check]
    I -->|gaps found, depth left| J[Refine queries] --> B
    I -->|sufficient| K[Answer + citations + grounding]

5.4 Adapters & their acquisition ladders

Adapters escalate through tiers only when needed — a static GET costs ~200 ms; a browser costs seconds.

🌐 web_deep — static → embedded → headless → stealth

flowchart LR
    L0["L0 · httpx + trafilatura<br/>~200ms"] --> J{needs JS?<br/>or near-empty?}
    J -->|no| OUT["Document"]
    J -->|yes| L15["L1.5 · __NEXT_DATA__ / JSON-LD<br/>zero browser cost"]
    L15 -->|found| OUT
    L15 -->|no| L2["L2 · Playwright Chromium<br/>network-idle + auto-scroll"]
    L2 --> L3["L3 · stealth · proxy · cookies<br/>(opt-in)"]
    L3 --> OUT

▶️ youtube — captions → ASR → diarization

Transcripts are cut into paragraphs on pauses, mapped onto chapters as sections, and every chunk is anchored to a timestamp — [12:04] resolves to youtu.be/ID?t=724.

📄 documents — PDF ladder + office + data cards

flowchart LR
    P0["P0 · text-layer check<br/>chars/page"] --> Q{scanned?}
    Q -->|no| P1["P1 · pypdfium2 text<br/>page-anchored"]
    Q -->|yes| P3["P3 · OCR (optional)"]
    P1 & P3 --> DOCX["DOCX / PPTX / EPUB → Markdown"]
    P1 --> TAB["CSV / XLSX → Data Card<br/>(schema · ranges · head)"]

Spreadsheets are never chunked as prose — they become a queryable Data Card (columns, dtypes, null %, ranges, sample rows).

🖼️ images — classify → route → VLM/OCR

flowchart LR
    I["image"] --> PH["pHash dedup"]
    PH --> CL{classify}
    CL -->|chart| VT["VLM → data table"]
    CL -->|diagram| VS["VLM → structure"]
    CL -->|screenshot| OV["OCR + VLM"]
    CL -->|photo| CAP["VLM caption"]
    VT & VS & OV & CAP --> TXT["content_md<br/>(text-retrievable)"]

Embedded images inside PDFs become child documents linked to their parent page — a chart on p.14 gets its own OCR/VLM description, retrievable by text query.

5.5 Anchors — what makes citation real

The difference between "RAG" and verifiable RAG:

5.6 Coverage & grounding — the differentiator

coverage(topic, filters) — what the corpus holds and where it's thin:

{
  "n_documents": 47,
  "by_source": { "youtube": 12, "web": 28, "document": 5, "image": 2 },
  "date_histogram": { "2024": 31, "2025": 14, "2026": 2 },
  "domain_concentration": 0.62,
  "authority": { "primary": 4, "secondary": 31, "unknown": 12 },
  "gaps": [
    "No source newer than 2025-12 — topic likely evolved",
    "62% of coverage from a single domain — low independence",
    "No primary/official documentation present"
  ]
}

Every ask() returns a grounding report — including unsupported_spans, the sentences that could not be entailed by the retrieved evidence:

{
  "grounding": {
    "chunks_used": 6, "sources_used": 4, "independent_domains": 3,
    "oldest_evidence": "2024-08-11", "newest_evidence": "2026-03-11",
    "unsupported_spans": ["Penguins architected the platform overnight."],
    "warnings": ["2 of 4 sources are stale (>180d)"]
  }
}

5.7 Quality scoring

One quality_score ∈ [0,1] per document — components stored separately so scores can be re-weighted without re-ingesting.

quality = 0.30 × acquisition_fidelity      # tier: T0=.95 T1=.70 T2=.90 · L0=.90 · P1=.95
        + 0.25 × extraction_completeness    # text ratio · table survival · no truncation
        + 0.20 × authority                  # primary source · domain · named author
        + 0.15 × recency_fit                # half-life decay vs. topic volatility
        + 0.10 × structure                  # headings · resolved sections · valid anchors

5.8 Project structure

anchor_ai/
├── core/          document · protocols · chunking · quality        (L1 keystone)
├── adapters/      youtube · web_deep · web_search · documents · images
├── corpus/        store · vectors · embeddings · retrieval · rerank
│                  blobs · dedup · freshness · coverage · grounding
│                  entailment · image_index                          (L0 durable value)
├── orchestration/ budget · queue · research · sync                  (L3)
├── interfaces/    cli · mcp_server                                  (L4)
└── bootstrap.py   first-run setup (browser + model)

5.9 Tech stack

5.10 Optional model backends

Everything above runs with zero external services. Heavier capabilities are opt-in extras, each with a graceful fallback:

pip install "anchorx[full]"     # everything below
pip install "anchorx[asr]"      # faster-whisper       — YouTube T2 ASR
pip install "anchorx[diarization]"  # pyannote.audio   — YouTube T3 speakers
pip install "anchorx[ocr]"      # pytesseract          — scanned docs / screenshots
pip install "anchorx[clip]"     # open-clip            — text ↔ image search
pip install "anchorx[stealth]"  # playwright-stealth   — web_deep L3

6. License

Released under the MIT License — see LICENSE for the full text.

MIT License · Copyright (c) 2026 Syed Muhammad Awais Gillani

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction … THE SOFTWARE IS PROVIDED "AS IS",
WITHOUT WARRANTY OF ANY KIND. See LICENSE for the complete text.