veccore

A ports-and-adapters vector substrate: embed → index → search → hydrate, backend- and embedder-agnostic.

veccore ships generic mechanisms and clean seams for semantic retrieval. You register a collection (choosing a backend + embedder), embed and upsert your items, and search — veccore returns {id, score, payload} and you hydrate the full record from your own system of record. Nothing about your domain lives in veccore; you never reinvent embedding, indexing, or idempotency.

pip install veccore                      # core (numpy only)
pip install 'veccore[qdrant]'            # + Qdrant server backend
pip install 'veccore[openai]'            # + OpenAI embedder (gated egress you inject)
pip install 'veccore[fastembed]'         # + on-device fastembed embedder (no API key, no network)
pip install 'veccore[st]'                # + sentence-transformers embedder

Quickstart

from veccore import (VecCore, CollectionRegistry, CollectionSpec, AuditEnvelope,
                     MockEmbedder, SqliteNumpyStore)

store = SqliteNumpyStore(path="/tmp/notes.sqlite")
core = VecCore(registry=CollectionRegistry(),
               stores={"sqlite_numpy": store},
               embedders={"mock": MockEmbedder()})
core.register(CollectionSpec(name="notes", backend="sqlite_numpy", embedder="mock", dim=64))

# every call carries an audit envelope: who + why (where/what are filled by veccore)
audit = AuditEnvelope(who="my_tool", why="index notes")
core.embed_upsert("notes",
                  [{"id": "n1", "text": "vector search over notes", "payload": {"src": "n1"}}],
                  audit)

hits = core.search("notes", "nearest ideas", k=5,
                   audit=AuditEnvelope(who="my_tool", why="find related"))
for h in hits:                           # each hit is {id, score, payload}
    ...                                  # hydrate the full record from YOUR store by h.id

What's in the box

• Core ports — EmbedderPort, VectorStorePort, and GatePort (injected, so veccore stays free of any spend/telemetry dependency — you hand it your gate, or NullGate for local/free embedders).
• Backends — SqliteNumpyStore (in-process, brute-force cosine; implements the optional matrix() analytics seam) and QdrantStore (server; implements search(filters=…) metadata/ACL push-down + Batch ingest). Future backends slot in by implementing VectorStorePort.
• Embedders — MockEmbedder (deterministic, zero-spend), OpenAIEmbedder (network, gated), LocalEmbedder (fastembed / sentence-transformers, on-device), and CompositeEmbedder (fuse several embedders so a plain dot product equals their weighted fused cosine).
• Content-sha idempotency — re-embed a row only when its text changes; a rerun over an unchanged corpus embeds nothing, and changing one item's text re-embeds exactly that one.
• Typed errors — Unavailable(code=…) and NotSupported(capability, backend) — never a silent degrade.
• Estimate-first — core.estimate(...) sizes a job (tokens + $) with zero spend before you embed.
• Batch ingest — BatchIngest for large corpora via a provider Batch API: chunked, resumable, a durable handle ledger, every result accounted; realtime and non-blocking collect both supported.
• Optional seams, negotiated — capabilities() + typed NotSupported mean expanding usage is additive: a consumer starts on search(collection, q, k), later adds filters=…, later calls matrix(), with no rewrite.

Design

veccore's line is mechanism vs policy: veccore ships the generic mechanisms + seams; each consumer builds its domain policy on top. The gate (spend/egress) and any observability sink are injected, so veccore has one hard dependency — numpy — and everything else (Qdrant, OpenAI, fastembed, sentence-transformers) is an optional extra you install only for the adapter you choose.

License

MIT © 2026 Ash Damle. Developed in the keelworks monorepo.