yapstack 0.2.0

Talk to your files. Local RAG that actually works — llama.cpp + turbovec.

YapCache

Your files been yapping? Finally, you can yap back.

YapCache is a local RAG pipeline that indexes your docs — PDFs, spreadsheets, meeting notes, images, voice recordings — and lets you ask stuff in plain English. No data leaves your machine. No API keys. No wifi needed. Pure ✨ local vibes ✨.

yap ingest ./docs
yap ask "What did the quarterly report say about revenue?"
yap ask "Summarise the investor deck" --stream
yap interactive

Quick start

# 1. Install
pip install yapstack[all]

# 2. Download a model (default: Llama 3.2 3B Q4)
huggingface-cli download bartowski/Llama-3.2-3B-Instruct-GGUF \
  Llama-3.2-3B-Instruct-Q4_K_M.gguf --local-dir ./models

# 3. Point it at your documents
cp ~/Documents/report.pdf ./data/

# 4. Ingest
yap ingest ./data

# 5. Ask
yap ask "What are the key findings?"
yap ask "Summarise the financials" --stream
yap interactive

One-shot setup script

bash setup.sh              # creates .venv, installs deps, downloads model
source .venv/bin/activate
yap ingest ./data
yap ask "your question"

Features

Multi-format parsing	PDF, DOCX, XLSX, CSV, images (EXIF + optional LLM captions), audio (Whisper), Markdown, HTML, code, plain text
Hybrid search	Semantic (cosine ANN via LanceDB) + keyword (BM25) fused with Reciprocal Rank Fusion
Query optimisation	HyDE (hypothetical document embedding), query rewriting, sub-question expansion
Cross-encoder reranking	ms-marco-MiniLM-L-6-v2 scores candidate chunks against the query
Streaming	Token-by-token output, interactive REPL mode
Deduplication	Re-running ingest only adds new content (MD5-based)
Sources	Every answer cites which files the information came from
Evaluation	NDCG, MRR, Recall, Precision for retrieval; BLEU, ROUGE-L, faithfulness, latency for answers
GPU auto-detect	Metal on macOS, CUDA on Linux, CPU fallback — no config needed
Cross-platform	macOS (Apple Silicon), Linux (NVIDIA), Windows (CPU)

CLI reference

yap [OPTIONS] COMMAND [ARGS]...

Commands:
  ingest       Parse, chunk, embed, index
  ask          Answer a question
  interactive  REPL session
  evaluate     Run eval harness
  status       Show index stats

yap ingest

yap ingest [DATA_DIR] [options]

Option	Default	Description
DATA_DIR	./data	Directory to ingest (recursive)
--index-dir	./index_store	Where to write the index
--model	config default	HF model spec (for image captioning)
--caption	off	Enable LLM image captioning (loads model, slower)

Ingest walks the directory, parses every supported file, splits text into sentence-boundary chunks, extracts keywords (KeyBERT), embeds via turbovec/sentence-transformers, writes to LanceDB + BM25. Deduplicates by MD5 hash — safe to re-run.

yap ask

yap ask "your question" [options]

Option	Default	Description
--index-dir	./index_store	Index to query
--model	config default	HF repo:file.gguf or local path to GGUF
--no-optimize	off	Skip HyDE / rewriting (faster, less accurate)
--no-rerank	off	Skip cross-encoder reranking (faster)
--stream	off	Stream output tokens as they're generated
--show-sources / --no-show-sources	on	Print source file paths

yap interactive

yap interactive [--index-dir PATH] [--model PATH]

Starts a REPL. Type exit or Ctrl-C to quit. Streaming is always on.

yap evaluate

yap evaluate [--dataset PATH] [--index-dir PATH] [--model PATH]

Runs retrieval and answer quality metrics against a JSONL dataset. See eval format below.

yap status

yap status [--index-dir PATH]

Prints chunk count, BM25 document count, embedding dimension, and model path.

Install

pip

pip install yapstack[all]

Platform extras (auto-detected at runtime, no flag needed):

Extra	Installs	Benefit
[all]	Everything	Recommended
[metal]	turbovec + mlx	3-5x faster embeddings on Apple Silicon
[embed]	sentence-transformers + keybert	Keyword extraction + reranking
[llm]	llama-cpp-python	LLM inference

GPU acceleration (llama.cpp build)

Default llama-cpp-python installs with CPU support. For GPU:

# macOS Metal
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python --force-reinstall

# Linux CUDA
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python --force-reinstall

The CLI auto-detects your GPU and sets n_gpu_layers accordingly — no code changes needed.

Models

Default: bartowski/Llama-3.2-3B-Instruct-GGUF (Q4_K_M, ~2 GB).

Override with any GGUF from HuggingFace:

yap ask "question" --model "author/model:file.q4_k_m.gguf"
yap ask "question" --model /path/to/local/model.gguf

The model is downloaded to ~/.cache/yapstack/models/ on first use.

Configuration

All settings live in YapConfig (pydantic). Override fields programmatically:

from yapstack.config import YapConfig

cfg = YapConfig(
    model="author/model:file.gguf",
    chunk_size=384,
    top_k_rerank=10,
    temperature=0.3,
)

Field	Default	Description
model	bartowski/...:...Q4_K_M.gguf	HF repo:file or local path
chunk_size	512	Words per chunk
chunk_overlap	64	Overlap words between chunks
embed_model	nomic-ai/nomic-embed-text-v1.5	Sentence transformer for embeddings
embed_dim	768	Embedding dimension
matryoshka_dim	None	Set 256 for 3x smaller index
top_k_semantic	20	Raw semantic candidates
top_k_bm25	20	Raw keyword candidates
top_k_rerank	5	Final reranked results
rrf_k	60	RRF smoothing constant
n_ctx	4096	LLM context window
temperature	0.1	Generation temperature
max_tokens	1024	Max answer length
context_token_budget	2048	Tokens reserved for retrieved context

Pipeline overview

Data folder → Parsers (PDF, DOCX, images, audio, tables, text)
                → Sentence chunker (512w, 64w overlap)
                    → Keyword extractor (KeyBERT)
                        → Embedder (turbovec / sentence-transformers)
                            → LanceDB vector store + BM25 index

At query time:
  Query → HyDE + Rewrite + Expand
            → Semantic search + BM25 search
                → RRF fusion
                    → Cross-encoder reranker
                        → Context builder (token budget)
                            → LLM (llama.cpp) → Answer + sources

Eval format

Create eval/dataset.jsonl:

{"query": "What is the main topic?", "relevant_ids": ["uuid-1", "uuid-2"], "reference": "The main topic is..."}
{"query": "Who authored the report?", "relevant_ids": ["uuid-3"], "reference": "The report was authored by..."}

yap evaluate

Output:

── Retrieval Metrics ──────────────────────
  ndcg@5               0.7842
  mrr                  0.8100
  recall@5             0.7200

── Answer Metrics ──────────────────────────
  bleu (mean)          0.3214
  rouge-l f1 (mean)    0.5631
  faithfulness (mean)  0.8900
  latency p50          3.21s

Project layout

yapstack/
├── pyproject.toml           # Build config + deps
├── setup.sh                 # One-shot setup script
├── README.md
│
├── src/yapstack/
│   ├── cli.py               # Click CLI (ingest / ask / interactive / evaluate / status)
│   ├── config.py             # Pydantic YapConfig
│   ├── _model.py             # HF model download + GPU auto-detect
│   │
│   ├── ingest/
│   │   ├── pipeline.py       # Orchestrator
│   │   ├── loader.py         # File discovery + parser routing
│   │   ├── chunker.py        # Sentence-boundary chunker
│   │   ├── metadata.py       # KeyBERT keyword extraction
│   │   └── parsers/          # PDF, DOCX, image, table, text, audio parsers
│   │
│   ├── index/
│   │   ├── embedder.py       # Auto-detect: turbovec → sentence-transformers
│   │   ├── vector_store.py   # LanceDB wrapper (HNSW cosine)
│   │   └── bm25_store.py     # rank-bm25 wrapper (persistent)
│   │
│   ├── retrieve/
│   │   ├── searcher.py       # Hybrid search orchestrator
│   │   ├── query_optimizer.py# HyDE + rewrite + sub-query expansion
│   │   ├── fusion.py         # RRF + cross-encoder reranker
│   │   └── fallback.py       # Low-confidence / no-result handling
│   │
│   ├── generate/
│   │   ├── llm.py            # llama-cpp-python wrapper (singleton, streaming)
│   │   ├── context_builder.py# Token-budget context packing
│   │   └── prompt_templates.py
│   │
│   └── eval/
│       ├── harness.py        # Full eval runner
│       ├── retrieval_eval.py # NDCG, MRR, Recall, Precision
│       └── answer_eval.py    # BLEU, ROUGE-L, faithfulness, latency
│
├── data/                     # Drop your documents here
├── index_store/              # LanceDB + BM25 index (auto-created)
├── models/                   # Downloaded GGUF files
└── eval/
    └── dataset.jsonl         # Your eval cases

Requirements

• Python >= 3.10
• macOS: Apple Silicon (M1–M4), macOS 13+
• Linux: NVIDIA GPU (CUDA) or CPU
• Windows: CPU only
• Disk: ~2 GB (model) + index size
• RAM: ~4 GB free at inference time