askfaro-embedded-search

Incremental hybrid retrieval with identical semantics on the server and on-device.

askfaro-embedded-search is a small, dependency-light library for searching a continuously growing collection of heterogeneous objects — notes, contacts, emails, tasks, tools, anything — without ever rebuilding an index. It is built for the pattern modern assistant apps need:

Index server-side, retrieve on-device. Embeddings are computed once, centrally. Each user's slice of the index is replicated into a local SQLite shard, and the device answers queries locally — fast, offline, and private — with exactly the same ranking the server would produce.

Built and dogfooded by Faro; also the embedded retrieval engine of Scope.

Why another retrieval library?

Most RAG tooling assumes a batch world: ingest a corpus, build a tree/graph/index, query it. Real applications have object-level CRUD — a contact edited, a task created, an email deleted — and small/on-device context windows that punish irrelevant results. askfaro-embedded-search makes two opinionated choices:

1. Incremental-first. One upsert per object write. Postgres (pgvector HNSW + tsvector GIN) and SQLite (FTS5 + vector blobs) both take per-row inserts natively, so there is no "rebuild" anywhere in the design. Hierarchical enrichment (summary and cluster nodes) lives in the same flat pool as extra rows — never on the write path.
2. Rank-based fusion. Lexical and semantic retrievers run in parallel and are fused with Reciprocal Rank Fusion (RRF, K=60). Because RRF consumes ranks, not raw scores, the incomparable scoring scales of ts_rank_cd (Postgres) and bm25() (SQLite FTS5) don't matter: the same corpus and query rank identically on both backends. That is what makes "same engine, server and device" honest rather than aspirational.

Quick start

from askfaro_embedded_search import IndexDoc, SearchIndex, OpenAICompatibleEmbedder
from askfaro_embedded_search.backends.sqlite import SQLiteBackend

index = SearchIndex(
    SQLiteBackend("search.db"),
    OpenAICompatibleEmbedder("https://api.openai.com/v1", api_key, "text-embedding-3-small"),
)

await index.upsert(IndexDoc(
    object_type="note", object_id="n1",
    title="Quantum entanglement notes",
    body="spooky action at a distance",
    partition="user-42",                      # isolation + shard key
))

results = await index.search("entanglement", partition="user-42", k=5)
# SearchResult(object_id='n1', match_type='hybrid', score=..., ...)

Deleting and updating are first-class — deletes are tombstones so they propagate through shard sync:

await index.delete("note", "n1")

Filtering

Filter by partition (the isolation/shard key), object type, node kind, or arbitrary structured attributes:

await index.upsert(IndexDoc(
    object_type="tool", object_id="stripe/refund",
    title="Refund a charge", body="...",
    attrs={"category": "finance", "status": "active"},   # structured filter fields
))

# Only finance tools:
await index.search("refund a customer", object_types=["tool"], attrs={"category": "finance"})

attrs is matched by containment (all given keys must equal), backed by a JSONB GIN index on Postgres and json_extract on SQLite — so it filters identically on server and device.

Keyword channel

The prose that makes body good for an agent to read (and to embed well) is often not the text that makes it findable by keyword — distinctive terms, synonyms, and codes get diluted in a sentence. keywords is a separate lexical channel: it feeds the BM25/FTS index but is kept out of the embedding, so it sharpens keyword recall without polluting the semantic vector. It is never returned to callers (display uses payload).

await index.upsert(IndexDoc(
    object_type="doc", object_id="ar-guide",
    title="Invoicing", body="How to send a bill and chase payment.",
    keywords=["dunning", "accounts receivable", "AR", "overdue"],   # lexical only
))
await index.search("dunning")   # matches via the keyword channel; the body never says "dunning"

Search modes

search(..., mode=...) trades precision for exploration by reweighting the lexical/semantic fusion (and the semantic floor):

await index.search("oauth redirect", mode="precision")   # exact-match leaning
await index.search("things like oauth", mode="explore")  # associative recall

Partitions

partition is the per-tenant isolation key (and the unit of on-device shard replication). It is optional and defaults to None, and a query's partition filter is an exact match: search(..., partition="user-42") returns only rows upserted with partition="user-42". Two consequences worth knowing up front, because both fail quietly rather than with an error:

• A row indexed without a partition (None) will not appear in a query that filters by a partition, and vice-versa. Pick one convention per corpus — either always set a partition, or never — and don't mix.
• search() with no partition does not scope to a tenant; it searches across every partition. Pass the partition on every query in a multi-tenant index.

Server-side: Postgres

from askfaro_embedded_search.backends.postgres import PostgresBackend  # pip install askfaro-embedded-search[postgres]

backend = PostgresBackend("postgresql+asyncpg://...", table="faro_embedded_search_index", dim=1536)
await backend.create_schema()     # idempotent; or transcribe the DDL into your migrations
index = SearchIndex(backend, embedder)

On-device: replicate a shard, query locally

from askfaro_embedded_search import export_shard, replicate

# Full export of one user's partition into a SQLite file:
shard = await export_shard(server_backend, "user-42.db", partition="user-42")

# Later, incremental delta sync (inserts, updates, AND deletes):
cursor = await replicate(server_backend, shard, partition="user-42", cursor=cursor)

The shard is a plain SQLite file whose schema is the interchange format — any runtime that can read SQLite (a future Swift/Kotlin reader, for instance) can retrieve against it. Embeddings travel with the shard; the device never needs to re-embed the corpus. Query embedding on-device can use a local model, a cached vector, or one tiny server round-trip for the query alone.

Tiering without batch trees

RAPTOR-style hierarchies buy small-context windows a lot — but a recursive tree can't be rebuilt on every insert. askfaro-embedded-search keeps the index flat and gets the benefit through node kinds:

• leaf — the object itself (default).
• summary — an optional one-line abstract of an object, indexed alongside it. O(1) per object, generated on your write path or a background pass.
• cluster — optional theme summaries produced by a periodic background sweep over existing embeddings.

All kinds live in the same pool and are retrieved by the same top-k ("collapsed" retrieval); by default multiple hits on one object collapse into its best row, with matched_node_kinds telling you which handles matched:

await index.upsert_many([
    IndexDoc(object_type="note", object_id="n9", title="Meeting notes", body=transcript),
    IndexDoc(object_type="note", object_id="n9", node_kind="summary",
             title="Summary: hiring sync", body="decided to open two roles"),
])

Multiple embedding spaces (dual-model / on-device)

A row can carry vectors from more than one model — typically a high-quality server model and a smaller on-device model — each as its own independently-queried space. Vectors from different models are never compared, so there's no mismatch.

index = SearchIndex(
    backend,                                    # configured with both spaces
    embedders={"server": openai, "local": on_device_minilm},
    default_space="server",
)

# A note participates in both spaces (default); an email is server-only:
await index.upsert_many([
    IndexDoc(object_type="note",  object_id="n1", title="...", body="..."),
    IndexDoc(object_type="email", object_id="e1", title="...", body="...",
             embed_spaces=["server"]),         # no on-device vector
])

# On the server, query the server space; on-device, query the local space:
await index.search("quarterly plan", space="server")   # web/server
await index.search("quarterly plan", space="local")    # device shard

The device shard carries only the on-device space (smaller — no server vectors):

shard = await export_shard(server_backend, "user-42.db",
                           partition="user-42", spaces=("local",))

Because a server-only object (the email above) has no local vector, on-device it's keyword-searchable but not semantic — its text still syncs, so FTS finds it offline; semantic search on it requires querying the server. This falls out of the per-space null-vector handling, no special-casing. Configure backends with matching spaces: PostgresBackend(..., spaces={"server": 1536, "local": 384}), SQLiteBackend(..., spaces=("local",)).

Heterogeneous objects

Per-type behavior lives in code, not schema, via a tiny registry:

from askfaro_embedded_search import register, docs_for

@register("contact")
def index_contact(c) -> IndexDoc:
    return IndexDoc(object_type="contact", object_id=str(c.id),
                    title=c.name, body=f"{c.role} at {c.company}",
                    payload={"avatar": c.avatar_url})

await index.upsert_many(docs_for("contact", some_contact))

payload is carried into results (and shards), so result lists render without joining back to your application database.

Design notes

• Embedding failure is non-fatal, but never silent. A row written without a vector still serves lexical queries and gains semantic retrieval after a backfill — availability over completeness. Every embedding failure (at index time and query time) is logged at WARNING on the askfaro_embedded_search logger with the exception, so a misconfigured embedder (bad key, wrong endpoint) surfaces instead of silently degrading search to keyword-only. Wire that logger up in your app to catch it.
• Exact semantic scan on SQLite. Per-user shards are small (tens of thousands of rows); an exact cosine scan (numpy-accelerated when present) costs no index maintenance and returns exact results. ANN acceleration (e.g. sqlite-vec) can be added without changing the file format.
• Diversity, not padding. An optional per-group cap (diversity_key) drops near-duplicate siblings instead of deferring them.
• Stemming parity. Postgres uses the english text-search config; SQLite FTS5 uses the porter tokenizer. Both stem morphological variants ("groceries" → "grocery") so the server and a device shard rank the same corpus identically.
• No framework. Plain SQL on both backends, zero required dependencies in the core.

Requirements

• Python 3.11+. The core (SQLite backend) needs only the standard library — SQLite's FTS5 and JSON1 extensions ship with CPython's bundled SQLite.
• Postgres backend ([postgres] extra) needs a database with the pgvector extension available. create_schema() runs CREATE EXTENSION IF NOT EXISTS vector and the rest of the DDL in one idempotent call (or transcribe it into your own migrations). It is safe to re-run; it also adds new columns in place when you upgrade the library.

Troubleshooting

The library raises clear, actionable errors for the common setup mistakes (all subclasses of FaroSearchError):

Other expected behaviours (not errors): searching a space with no embedder returns lexical-only results, and a doc indexed without a vector still serves keyword search — both by design (availability over completeness).

Installation

pip install askfaro-embedded-search                # core (SQLite backend, stdlib only)
pip install askfaro-embedded-search[postgres]      # + Postgres/pgvector backend
pip install askfaro-embedded-search[http]          # + OpenAI-compatible embedder
pip install askfaro-embedded-search[numpy]         # + fast cosine on SQLite

License

MIT