silica-core 0.8.1

Retrieval tools for coding agents. No model in the loop.

Point it at a folder and the agent you already use gets five tools that return located evidence, never answers.

Silica works over the markdown, code, PDFs and office files under one root and serves them to Claude Code, Cursor, Zed, Codex and every other harness it knows, as MCP tools or as shell commands that print the same JSON. A hit is a path, a section, a line, a window of text and the numbers to judge it by. Silica never answers, summarises, plans or remembers for you: the harness owns the loop.

Install

uv tool install 'silica-core[mcp]'

pipx install 'silica-core[mcp]' works the same. The package is silica-core, the command is silica, the tools are silica_*: the distribution carries the name, the code keeps the namespace.

Quickstart

In any folder of markdown, code, PDFs or office files:

silica init                               # adopt the folder and build the first index
silica search "leveled compaction" -k 5  # return the best located passages
silica setup claude                      # register the MCP server, write the guidance block into ~/.claude/CLAUDE.md

Nine arXiv papers in a folder, indexed in 2.1 s. The hit names the file, the page and the passage, and the page beside it is the check, not a promise.

Benchmarks

Silica's closest comparisons are zvec-grep and ck: local retrieval tools over files. Each arm received the same BEIR SciFact and NFCorpus documents and queries. Scores are nDCG@10 (higher is better), shown as SciFact · NFCorpus.

Silica's lexical index needs no model; the competitors serve lexical search from an index that also includes embeddings. BEIR's published BM25 baselines are 0.665 · 0.325. On the hybrid row Silica and zvec-grep tie at the 95% interval: the same vectors rank the same, with no daemon, no vector store and no bundled model. See benchmarks for corpora, methods, intervals and commands.

On code, grep then silica_code_pack as the only reader was measured against grep and file reads: twelve repository questions with Haiku 4.5, the four hardest again with Sonnet 5, and no task separated the two arms on the answer, as none did with Opus 5 the day before. Where the answer is one symbol's body in a file not worth reading whole the pack halves the cost, two constants in two files at $0.067 against $0.034 and a twelve-line label function at $0.054 against $0.033, and on the local task Sonnet pays a third less with it ($0.200 against $0.134) because it finds the symbol inside the pack. Numbers per model and per task in benchmarks.

Searching code, not only reading it: with the code index on (SILICA_INDEX_CODE, one unit per function, method, class or constant), Silica ranks the answering file ahead of zvec-grep on zvec-grep's own twenty SWE-QA questions with the same embedder, file MRR 0.68 against 0.54, the interval clear of zero, and ties on twelve local ones. Asked for the way the plugin now asks (an imperative description, a prompt hook, a CLAUDE.md block), Sonnet ran the search in 17 of 20 runs and closed the question in 4.7 turns instead of 6.5, at the same cost, the judge up nine points inside its interval. Numbers in benchmarks.

Tools

Search indexes markdown and the PDFs that carry a text layer, and, with SILICA_INDEX_CODE set, source files one unit per function, method, class or constant: a hit's section is the symbol, span its lines, and silica_read(path, section=…) serves the body. Without the switch a .py is excluded in silica_files and served by silica_read by line. For a symbol whose name is known, grep wins; for a question that names none, the search comes first, and the plugin's prompt hook asks the model to say so before it greps (measured 2026-09-10: with the ask in place the search ran in 17 of 20 runs). The contract, the reply shapes and the acceptance checks are in TOOLS.md; nothing there needs an API key, a model or a network.

How search says no

A ranked list always has a top, even when the corpus does not answer. Three fields show how much the result is worth:

• coverage: the share of the query's idf mass the hit's matched terms carry. Near 1, every rare term matched; near 0, only common words did.
• terms_absent: query terms that occur nowhere in the corpus.
• matched_terms: which words this hit actually contains.

raft consensus log replication over the same nine papers: raft and consensus occur in none of them, coverage falls to 0.19, and the top hit is about data replication, not Raft. Silica exposes the signals and lets the harness decide whether to stop, read or rephrase.

Ranking is BM25 over documents, then over the heading sections of the top documents, returning each hit's densest window. --also QUERY ranks another query group separately and fuses the results, useful for searching a concept and its exact identifiers together. Searches refresh small changes inline; larger ones return stale with pending until silica index catches up, and interrupted builds resume. Use <root>/.silicaignore to exclude content.

The dense leg

Optional section embeddings catch paraphrases that share no rare words with the answer. After they are built, every search fuses them with BM25. Use any OpenAI-compatible /v1/embeddings endpoint:

ollama pull nomic-embed-text
export SILICA_EMBEDDING_BASE_URL=http://localhost:11434/v1 SILICA_EMBEDDING_MODEL=nomic-embed-text
export SILICA_EMBEDDING_DOC_PREFIX='search_document: ' SILICA_EMBEDDING_QUERY_PREFIX='search_query: '  # what nomic's card asks for
silica index --embed

Or use a local static model, with no server:

uv tool install 'silica-core[mcp,dense]'
export SILICA_EMBEDDING_MODEL=model2vec/minishlab/potion-retrieval-32M
silica index --embed

Text leaves the machine only for an endpoint that is not on it, and only after silica index --embed --allow-remote grants that host once. Until then search stays lexical. The reply always says whether the dense leg ran.

What it reads

• PDF with a text layer: searched directly, one section per page; no conversion or sidecar.
• Scanned PDF, image, PPTX, XLSX: silica import uses MinerU to create a guarded .md sidecar. Silica reports it as stale if the source changes.
• DOCX, EPUB, FB2, RTF, XLS, ODF: converted with no extra.
• Audio and video: ffmpeg plus SILICA_STT_BASE_URL.
• CSV and other data files: readable by line but excluded from search.

silica doctor says which lanes this machine has. Extras: [connect] adds the Obsidian bridge, [dense] the local embedder, [all] everything.

Harnesses

silica setup <client> writes the registration into the client's own config. It backs up existing config and refuses malformed files; for claude it also puts a guidance block between markers into ~/.claude/CLAUDE.md, when to search before grep, replaced on a second run and nothing else touched. silica setup --list shows paths for claude, codex, cursor, windsurf, zed, cline, roo, continue, goose, opencode, openhands, gemini, dsh, hermes, openclaw, agent-zero, claude-desktop, lmstudio, anythingllm and librechat; shell, python and generic print recipes for anything else. By default the server serves the folder the client opens in; use --vault DIR or SILICA_VAULT for a fixed root.

npx skills add kiycoh/silica-core installs the skill that tells an agent when to reach for the tools. It installs nothing else: on its own it leaves an agent holding instructions for tools that are not there. Setup details, shell recipes, Docker and the optional REPL are in docs/harnesses.md.

Other surfaces

• Extended tools: silica mcp --extended adds the wikilink tools, silica_links, silica_backlinks, silica_orphans and silica_unresolved, over the same root.
• Obsidian: silica connect hosts the bridge the Obsidian plugin dials into, so writes land through the vault API while the app is open. The five core tools do not need it.
• REPL: silica repl runs a small reference agent over the same tools. It is the only surface that needs a model (SILICA_MODEL).

The core keeps no memory lane, prompt injection, summaries or undo journal. Undo is git.

License

MIT. See LICENSE.