vd

One Pythonic interface to ~15 vector databases. Store vectors, search them, filter by metadata, and switch between databases (memory, Chroma, Qdrant, pgvector, LanceDB, Pinecone, …) by changing one word.

Still typing code with your own ten fingers? Your artisanal corner is at the bottom.

For AI agents

Skills. vd ships agent skills in vd/data/skills/. They install into any agent host with gh skill:

gh skill install i2mint/vd vd-quickstart --agent claude-code
gh skill install i2mint/vd vd-backend-choose
gh skill install i2mint/vd vd-setup-backend
gh skill install i2mint/vd vd-ingest
gh skill install i2mint/vd vd-search
gh skill install i2mint/vd vd-ops
gh skill install i2mint/vd vd-add-backend

They also ship inside the wheel, so after pip install vd you can link them without GitHub:

mkdir -p ~/.claude/skills
ln -s "$(python -c 'import vd; print(vd.skills_dir())')"/vd-* ~/.claude/skills/

Project instructions for coding agents working on vd live in .claude/CLAUDE.md: architecture, test commands, conventions and the refactor roadmap.

What an agent can do with vd: recommend and set up a vector database for a user, index their documents, run filtered or hybrid searches, and move a collection between databases. The whole surface is two Mappings plus search:

import vd

client = vd.connect("memory")  # switch DB = change this one word
col = client.create_collection("docs")
col["a"] = vd.Document(
    id="a", text="cats", vector=[0.1, 0.9, 0.0], metadata={"kind": "pet"}
)
col["b"] = vd.Document(
    id="b", text="pizza", vector=[0.9, 0.0, 0.1], metadata={"kind": "food"}
)

for hit in col.search([0.1, 0.8, 0.0], limit=2):
    print(hit["id"], round(hit["score"], 3))
# a 1.0
# b 0.123

Install

pip install vd                    # core (stdlib + pyyaml) and the memory backend
pip install "vd[chroma]"          # plus one backend's client
pip install "vd[embedded]"        # plus chroma, qdrant, faiss, lancedb, sqlite_vec, duckdb
pip install "vd[all-backends]"    # plus every backend client

Each backend's client is an optional extra named after the backend. Keep the quotes: zsh treats [...] as a glob.

The mental model

vd stores and searches vectors. Turning text into vectors, embedding, is deliberately external: vd never embeds on its own. Hand it Documents that carry a vector, and search with a query vector.

For convenience, pass any text -> vector function as embedder, and raw text then works too:

import vd


def embed(text):  # stand-in; use a real embedding model
    return [text.count(c) / (len(text) or 1) for c in "aeiou"]


col = vd.connect("memory", embedder=embed).create_collection("notes")
col["k1"] = "cats and kittens"  # embedded for you
col["k2"] = ("dogs and puppies", {"kind": "dog"})  # text + metadata
print([h["id"] for h in col.search("a kitten", limit=1)])
# ['k1']

Without an embedder, passing text raises EmbeddingRequiredError: loud, never a silent wrong-model embedding.

The API

import vd

col = vd.connect("memory").create_collection("api")
col["k"] = vd.Document(id="k", text="hello", vector=[1.0, 0.0], metadata={"year": 2024})
doc = col["k"]  # get
print("k" in col, len(col), list(col))  # membership, count, keys
# True 1 ['k']
del col["k"]  # delete

search(query, *, limit=10, filter=None, egress=None) yields dicts {"id", "text", "score", "metadata"} where score is higher-is-better. Transform results with an egress such as vd.id_only, vd.id_and_score, vd.text_only, vd.id_text_score, or your own function.

Metadata filtering

One backend-agnostic, MongoDB-style filter language: $eq $ne $gt $gte $lt $lte $in $nin $exists $and $or $not.

import vd

col = vd.connect("memory").create_collection("posts")
col["a"] = vd.Document(
    id="a", text="", vector=[1.0, 0.0], metadata={"year": 2023, "kind": "news"}
)
col["b"] = vd.Document(
    id="b", text="", vector=[0.9, 0.1], metadata={"year": 2019, "kind": "blog"}
)
hits = col.search(
    [1.0, 0.0], filter={"year": {"$gte": 2020}, "kind": {"$in": ["news", "blog"]}}
)
print([h["id"] for h in hits])
# ['a']

Each backend declares which operators it honors. An unsupported one raises UnsupportedFilterError instead of silently mis-filtering. Backends with rich native filtering (Qdrant, Pinecone, MongoDB) translate the filter; the rest apply it client-side with the same semantics.

Hybrid search

vd.hybrid_search fuses a vector search with a keyword (BM25) search using Reciprocal Rank Fusion, on any backend:

import vd

col = vd.connect("memory").create_collection("hybrid")
col["a"] = vd.Document(id="a", text="error code E1234 in the parser", vector=[0.2, 0.8])
col["b"] = vd.Document(id="b", text="parser overview", vector=[0.9, 0.1])
hits = vd.hybrid_search(col, [0.9, 0.1], query_text="E1234", limit=2)
print([h["id"] for h in hits])
# ['a', 'b']

Weaviate, Elasticsearch, Redis and LanceDB run the keyword side on their own text index (isinstance(col, vd.SupportsHybrid)); other backends use a client-side BM25 scan.

Async

import asyncio
import vd


async def main():
    async with await vd.connect_async("memory") as client:
        col = await client.create_collection("docs", dimension=2)
        await col.set("a", vd.Document(id="a", text="x", vector=[1.0, 0.0]))
        return [h["id"] async for h in col.search([1.0, 0.0], limit=1)]


print(asyncio.run(main()))
# ['a']

Every backend works through a thread-pool wrapper. Backends listed by vd.list_async_backends() can return a native async client that does real non-blocking I/O: currently qdrant when connected to a server with url=. Check client.native_async.

Escape hatches

The facade never traps you. client.client is the raw backend client and collection.native is the raw backend collection. Both are supported, documented API for reaching backend-specific features.

Choosing and setting up a backend

import vd

vd.print_recommendation(
    corpus_size="medium",
    persistence=True,
    can_run_docker=True,
    cloud_ok=True,
    budget="free",
    needs_hybrid=False,
)
vd.print_backends_table()  # the whole landscape
vd.compare_backends(["chroma", "qdrant", "pgvector"])

vd.check_requirements("qdrant")  # diagnoses readiness and prints the next step
print(vd.setup_guide("qdrant"))  # pip / docker / env-var playbook
vd.install_command("qdrant")  # 'pip install "vd[qdrant]"'

check_requirements is deployment-aware. It checks the client library for embedded backends, whether a server answers for self-hosted ones, and the required environment variables for managed ones, and always ends with one concrete next action.

vd.list_backends() shows which adapters are installed and ready now.

The toolkit

Beyond the facade, vd bundles the composite operations people actually do:

• vd.search: multi_query_search, reciprocal_rank_fusion, hybrid_search, BM25Index, search_similar_to_document, deduplicate_results.
• vd.io: export_collection / import_collection (JSONL, JSON, directory).
• vd.migration: migrate_collection, migrate_client, copy_collection to move data between any two backends.
• vd.analytics: collection_stats, find_duplicates, find_outliers, validate_collection.
• vd.health: health_check_backend, benchmark_search.
• vd.text: convenience text cleaning and chunking.
• vd.TimeIndexedCollection: a time-windowed wrapper over any collection.
• CLI: vd backends, vd install, vd export, vd import, vd migrate, …

For artisanal, hand-typed contributions

Welcome, fellow keyboard enthusiast. Here is what the sections above don't already cover.

Dev setup and tests.

git clone https://github.com/i2mint/vd && cd vd
uv venv .venv && . .venv/bin/activate
uv pip install -e ".[test,dev]"
python -m pytest --doctest-modules -o doctest_optionflags='ELLIPSIS IGNORE_EXCEPTION_DETAIL'

That is what CI runs (CI also installs the ci-backends extra); package doctests are included. The backend-parametrized suites skip server backends that aren't running. Every server backend except turbopuffer runs locally without an account, Pinecone and MongoDB Atlas included, through their official emulators:

DOCKERHUB_MIRROR=mirror.gcr.io docker compose -f tests/docker-compose.yml up -d
uv pip install -e ".[test,dev,pgvector,redis,elasticsearch,weaviate,mongodb,milvus,pinecone]" "pinecone<10"
python -m pytest

See tests/README.md for connection settings and caveats.

Design rationale.

• Embedding is external. The core operates on vectors; an embedder is an injected, optional convenience, never a hard dependency.
• Two mappings. A Client is a Mapping of collections; a Collection is a MutableMapping of documents plus search. Idiomatic, minimal, familiar.
• Thin adapters. AbstractClient / AbstractCollection implement everything users see; a backend supplies a handful of raw primitives. Adding a backend is about 150 lines, and the vd-add-backend skill walks through it.
• Capabilities, not a fat base. Optional features (SupportsBatch, SupportsHybrid, SupportsNativeAsync) are @runtime_checkable protocols you feature-discover.

The longer design notes are in misc/docs/vd_design_notes.md, and the backend-selection research report is in misc/docs/.

Contributing. Open an issue or a pull request at github.com/i2mint/vd. Releases are automated: merging to master bumps the version and publishes to PyPI.

License

MIT