duckdb-vectors

A frugal, embeddable semantic-search building block for agentic systems — hybrid vector + exact-token search over a local DuckDB file, no external vector database.

duckdb-vectors is a python package that facilitates the implementation of small-scale semantic search into any kind of agentic system, in 3 steps:

1. chunk items and store their embeddings in a DuckDB database
2. bake the database into the container that packages your agent
3. search semantically using vector distances and BM25 in tens of milliseconds

Use cases

duckdb-vectors is a nimble building block that has been designed for following use cases:

• Retrieval-Augmented Generation (RAG) powered by thousands of static documents — You do not see the need for an external vector store or knowledge base. You prefer a composable toolchain over fat frameworks. You are looking for extremely fast responses. Turn an existing markdown folder into a RAG agentic capability.

• Select skills exposed to your agents — The toolbox of your agent is growing exponentially. You create a tool for selecting appropriate skills instead of exposing all of them. In addition, each tool comes with attributes that should be considered during the search. Add a registry of hundreds of skills selected by semantic search.

• Chat with your LLM-wiki — You have setup an agentic knowledge base that is managed and enhanced incrementally. It consists of a growing set of Markdown pages, that deserves careful chunking and embedding for response relevance. You want a simple complementary toolchain for chatting with your Wiki. Leverage an LLM-managed wiki into an agent.

Quick start

Get duckdb-vectors from the PyPI repository:

$ pip install duckdb-vectors             # install once, batteries included

Indexing a set of Markdown pages directly from the command-line. This produces a single DuckDB file:

$ duckdb-vectors index ./docs            # index a folder of markdown
$ duckdb-vectors report                  # report on database content
$ duckdb-vectors search "my question"    # hybrid search

You can also index content dynamically from the harnessing code. This is required if the agent gets new skills dynamically for example.

Then perform semantic search from within your python code:

from duckdb_vectors import VectorStore

store = VectorStore()

for hit in store.query("how does an embedded database work?", top_k=3):
    print(hit.score, hit.chunk.chunk_id, hit.chunk.text)

Next steps: follow a step-by-step procedure under Documentation to build a RAG answering machine, chat with an LLM-wiki, or add a semantic skills registry, or copy a runnable example from examples/.

Customization

This section marks the boundaries of the package: what you can adapt out of the box, and where the extension points are for builders who want to contribute.

The default configuration of duckdb-vectors makes it workable out of the box. But you do have multiple options for adapting it to your specific requirements:

• Choose your harnessing framework, or create your own. duckdb-vectors focuses on semantic search. Pass a string, and it returns matching chunks. Use them as you want.

• Frontmatter headers of Markdown pages are injected as JSON objects. Each attribute can be considered in SELECT statements in addition to semantic search. Metadata is returned along with each chunk.

• Use a local embedding model, or a remote one. Titan from Amazon Bedrock is built-in. You can expect the registry of available embeddings to grow over time.

• Implement your own chunking class, that is fully adapted to your knowledge base. We have focused on Markdown files. You may have other kinds of documents to ingest.

Adding an embedding backend or a chunker is a great first contribution — the registries are built for it. See CONTRIBUTING.md to get started.

Documentation

Pick the entry point that matches where you are:

• New to duckdb-vectors? Read the primitive for understanding the architecture and building blocks, then the guide for the straightforward developer workflow.

• Building a specific capability? Follow a step-by-step procedure: RAG answering machine, LLM wiki to RAG, or semantic skills search.

• New to DuckDB? See the DuckDB workbook.

• Packaging an agent? See the Docker agent guide and its FAQ.

• Deploying on AWS? See the Bedrock AgentCore guide and FAQ, or the Lambda guide and FAQ.

• Contributing or developing? Start with CONTRIBUTING.md, then the developer workbook for prerequisites, setup, and the full workflow.

Configuration

Settings resolve with the precedence: explicit argument > DUCKDB_VECTORS_* environment variable > default.

Embedding backends

The selection of a backend for embeddings is resolved from a string. This can be a bare name (local) or a name:variant pair (bedrock:titan). local is the default and the minimum viable setup for end users. The registry is contributor-extensible — register open or closed, local or cloud backends without touching the core:

from duckdb_vectors import register_embedder

register_embedder("mybackend", lambda variant: MyEmbedder(variant))

Here is the list of embeddings that are currently available:

Chunking backends

A chunker is resolved from a string: a bare name (markdown) or a name:variant pair. markdown is the default and the minimum viable setup for end users. The registry is contributor-extensible — register your own splitter without touching the core:

from duckdb_vectors import register_chunker

register_chunker("mychunker", lambda variant: MyChunker(variant))

Metadata-aware search (injectable ranking SQL)

duckdb-vectors has built-in SQL statements for searching in DuckDB database. This is good enough for common cases. In addition, you can provide your own SQL statements for refining to your exact needs.

For example:

• Adapt the search to a given audience. You want to serve both customers and your staff from the same agent. The distinction is carried by the JWT token that reaches the agent. The searching code can extract either "customer" or "staff" from the token, and inject this in the SQL statement, so that only the matching set of pages is considered.

• Tweak the ranking algorithm. You want to decay progressively the aging documentation pages, so that fresh content gets better ranking than oldies. The searching code can integrate the modification date from meta-attributes and decay the basic ranking accordingly.

Each ranking signal — vector and BM25 — runs a DuckDB SELECT that returns chunk_id best-first. The store collects the ids and fuses them with RRF. You can override either SELECT to restrict in-scope chunks (a WHERE over metadata) or to apply a decay factor on aging documents (a reshaped ORDER BY). When you provide no override, built-in defaults reproduce the standard behavior.

Templates use these substituted tokens: {table}, {distance} (vector only), {bm25} (text only), {top_k}. The search text is available as the bound parameter $q. Standard columns are in scope: metadata, source, title, stamped_at, text.

Overrides can be set once (shared across queries) via Settings or DUCKDB_VECTORS_VECTOR_SQL / DUCKDB_VECTORS_TEXT_SQL, or passed per call. Precedence is per-call > settings > default. A custom SELECT owns its own WHERE and supersedes the convenience filter dict for that signal.

Here is an example where the search is limited to policy pages, aka, where kind: policy:

scoped = (
    "SELECT chunk_id FROM {table} "
    "WHERE json_extract_string(metadata, '$.kind') = 'policy' "
    "ORDER BY {distance} LIMIT {top_k}"
)
store.query("retention rules", vector_sql=scoped, text_sql=scoped)

Here is a example for decaying aging documents (older chunks sink; age_days lives in metadata):

decayed = (
    "SELECT chunk_id FROM {table} "
    "ORDER BY {distance} "
    "* (1 + CAST(json_extract_string(metadata, '$.age_days') AS DOUBLE) "
    "/ 365.0) "
    "LIMIT {top_k}"
)
store.query("latest guidance", hybrid=False, vector_sql=decayed)

The injected SELECT is caller-authored SQL run against your own database (opened read-only in the serving path); the query text and embedding are parameter-bound.

Design note: lexical search without stemming

Hybrid search here fuses semantic (vector) retrieval with a BM25 lexical signal, but the BM25 index is built deliberately with stemmer = 'none' and stopwords = 'none'. This is a considered choice, not an oversight.

Semantic search already covers meaning, paraphrase, and synonyms, and it does so across languages. Traditional stemmed full-text search (Porter stemming plus a per-language stopword list) adds compute and storage, ties the index to one language, and — in our prior benchmarks on comparable corpora — did not improve relevance enough to justify that cost once semantic retrieval was in place.

What vectors do not handle well is exact short tokens: acronyms, product codes, identifiers, and single-keyword queries. That is precisely where a lexical signal earns its keep. So the BM25 index is kept, but as a language-agnostic exact-token match: ESRS finds ESRS, while run does not conflate with running. The two signals are complementary — vectors for meaning, exact-token BM25 for the literal strings vectors miss.

A developer who wants classic stemmed full-text behavior can still have it: build their own FTS index and supply a custom text_sql (see above). The default simply does not impose stemming on everyone.

Why?

You are right, trillions of solutions exist for semantic search. So, why bother?

Because we are frugal.

• We won't embrace a fat framework, but limit our code base and dependencies to the strict minimum. We just want to install and import a dedicated PyPI package that does one thing well.

• We won't delegate via the network a search across some thousands of items. A local function call is by far the fastest path to relevant answers. DuckDB is a perfect fit for such in-process use case. See examples/skill_registry.py.

• We won't use stemming search at all. Semantic search for sentences and exact matches for keywords cover our needs.

• We won't externalize a read-only vector database or knowledge store for some thousands of items. This should be baked into the file system of the agent itself. DuckDB just needs a single database file, this is just what we need.

• We won't introduce separate baking for the code and for data, since modern agents rely on Docker images anyway. See examples/Dockerfile.

• We won't invest on non-differentiating features. Trusted harnessing will come from boring technologies.

We did our homework, as reflected in the Alternatives section. It appears, in the end, that no python package was available to cover our frugal needs.

Alternatives

duckdb-vectors is not the only way to put vectors in DuckDB. The projects below overlap with it; each is a reasonable choice if its shape fits your need better than ours. Our goal is narrow on purpose: a small, framework- agnostic, read-mostly utility that bakes an index into a container image and answers in tens of milliseconds, with pluggable embedding and chunking and an S3-Vectors-flavored surface. Where a project pulls in a framework, a service, or a broader scope, we point you to it rather than compete with it.

• llama-index-vector-stores-duckdb — the official LlamaIndex integration that stores embeddings in DuckDB. Great if you are already building on LlamaIndex, because it plugs into that data model, retrievers, and query engines. We stay framework-agnostic: no LlamaIndex Node/Document types to adopt and no framework runtime to carry into the container.

• vectorwrap — a universal wrapper that switches one vector API across Postgres, MySQL, SQLite, DuckDB, and ClickHouse. Pick it when backend portability is the point. We go the opposite way: one backend (DuckDB), chosen deliberately, so we can lean on DuckDB-specific features (ARRAY columns, injectable ranking SQL) instead of a lowest-common-denominator API. It also expects you to bring your own embedding function; we ship pluggable embedders and markdown chunking.

• microrag — the closest neighbour: a universal RAG library with ONNX-backed embeddings and DuckDB storage. If you want an end-to-end RAG library, it is a strong fit. We are not a RAG library — we deliberately stop at retrieval (store, index, search) and leave generation, prompting, and LLM orchestration to the harness, so the component composes into any framework rather than owning the pipeline.

• duckdb ai community extension — an extension that calls embedding and chat providers from inside SQL. Reach for it when you want embeddings generated in the database and don't mind the provider wiring living in SQL. We keep embedding in Python so the same code runs offline in CI and against a cloud provider in production without SQL-level credentials.

• sqlrooms-rag — prepares and queries DuckDB-stored embeddings for RAG, primarily for the SQLRooms framework (usable standalone too). A good fit inside that ecosystem; outside it, our standalone, dependency-light surface and container lifecycle are a closer match.

• raghilda — a DuckDB-backed RAG store (from Posit) that can run fully in memory and search with BM25 when no embedding model is configured. Pleasant for quick local experiments. Our focus is the frozen, read-only, image-baked serving path and an S3-Vectors-flavored API rather than an in-memory exploration store.

• dvs — a DuckDB + OpenAI vector similarity search service. Choose it if an OpenAI-tied service is what you want. We default to local, offline embeddings and treat cloud providers as an opt-in, keeping the base install slim and CI-friendly.

Contributing

Contributions are welcome — adding an embedding backend or a chunker is a great first pull request. See CONTRIBUTING.md for the development setup, project principles, and pull-request guidelines.

License

MIT.