dot-search

Add vector, keyword and exact search to the database tables you already have.

from dot_search import SearchEngine, TableConfig, EmbeddingConfig

engine = SearchEngine(db_url="postgresql+psycopg2://user:pass@localhost/mydb")

engine.index(
    TableConfig(table="articles", embeddings=[EmbeddingConfig(key="body", source_column="body", dimension=1536)]),
    batch_size=500,
)

for hit in engine.search("neural networks", "articles"):
    print(hit.id, hit.score)

dimension is the vector size your embedding model returns — 1536 for OpenAI's text-embedding-3-small, 1024 for Mistral's mistral-embed. It has to match, or indexing fails on the first batch.

Why dot-search

Adding semantic search to an existing application usually means standing up a separate vector database, then keeping it in sync with the tables that hold the real data — a second source of truth, a sync job, and a new failure mode.

dot-search takes the other route: it adds embedding columns and search indexes to your existing tables, in your existing database. Your rows stay where they are, joins keep working, and a search result is a row you already own. Vector, BM25 and exact matching can run together, with results merged through Reciprocal Rank Fusion.

Features

• Vector search on any text column, through pgvector
• BM25 keyword search, through ParadeDB pg_search
• Exact substring matching
• Hybrid search fusing several strategies via Reciprocal Rank Fusion
• Several independent indexes on the same table
• Row filtering with a parameterised Filter API — filter values are bound, so they can safely carry end-user input
• Declarative configuration, persisted so an index can be reopened later
• Batch or caller-driven indexing
• Pluggable embedders, defaulting to dot-inference
• SQLite backend for local development and testing

Requirements

PostgreSQL (production)

Requires pgvector and ParadeDB pg_search:

CREATE EXTENSION vector;
CREATE EXTENSION pg_search;

SQLite (development and testing)

Uses sqlite-vec and FTS5. Convenient for tests, but not intended for production.

Installation

pip install dot-search

Configuration

Embeddings are produced through dot-inference, which reads its settings from DOTI_-prefixed environment variables:

Works with any OpenAI-compatible API (OpenAI, Mistral, OpenRouter, vLLM, TGI, Ollama, …). See the dot-inference settings for the full list, or pass your own embedding function instead.

Quick start

from dot_search import (
    BM25Config,
    EmbeddingConfig,
    ExactConfig,
    Filter,
    SearchConfig,
    SearchEngine,
    TableConfig,
)

engine = SearchEngine(db_url="postgresql+psycopg2://user:pass@localhost/mydb")

# --- 1. Index a table with vector search ---
engine.index(
    TableConfig(
        table="articles",
        embeddings=[EmbeddingConfig(key="body", source_column="body", dimension=1536)],
    ),
    batch_size=500,
)

# --- 2. Search ---
for hit in engine.search("neural networks", "articles"):
    print(hit.id, hit.score)

# --- 3. Combine several strategies on the same table ---
engine.index(
    TableConfig(
        table="articles",
        embeddings=[
            EmbeddingConfig(key="body", source_column="body", dimension=1536),
            EmbeddingConfig(key="title", source_column="title", dimension=1536),
        ],
        bm25=[
            BM25Config(key="title_bm25", source_column="title"),
            BM25Config(key="body_bm25", source_column="body"),
        ],
        exact=[ExactConfig(key="name_exact", source_column="name")],
    ),
    batch_size=500,
)

# --- 4. Hybrid search with row filters and weight overrides ---
results = engine.search(
    "fermentation and gut health",
    "articles",
    SearchConfig(
        limit=10,
        where=[
            Filter(column="published_year", op="gte", value=2022),
            Filter(column="topic", op="eq", value="health"),
        ],
        weights={"body": 1.0, "title": 0.3, "title_bm25": 0.5, "body_bm25": 2.0},
    ),
)

# --- 5. Single-strategy search ---
results = engine.search("fermentation", "articles", SearchConfig(strategy="bm25"))
results = engine.search("Dupont", "articles", SearchConfig(strategy="exact"))

# --- 6. Several independent indexes on one table ---
engine.index(
    TableConfig(
        table="articles",
        index_id="article_titles",
        embeddings=[EmbeddingConfig(key="title_only", source_column="title", dimension=1536)],
    ),
    batch_size=500,
)
results = engine.search("gut health", "article_titles")

# --- 7. Index specific rows (caller-driven batching) ---
engine.index(
    TableConfig(
        table="articles",
        embeddings=[EmbeddingConfig(key="body", source_column="body", dimension=1536)],
    ),
    row_ids=[1, 5, 10],
)

Search strategies

min_score

SearchConfig.min_score drops results scoring below a threshold. It is only meaningful for single-strategy searches (vector, bm25, exact), where raw scores are preserved and interpretable — a cosine similarity, for instance. It is rejected in hybrid mode, because RRF scores are rank-based and carry no absolute meaning; use limit there instead.

Filtering

Row filters are expressed with Filter objects rather than SQL fragments:

SearchConfig(where=[
    Filter(column="status", op="eq", value="published"),
    Filter(column="lang", op="in", value=["fr", "en"]),
    Filter(column="deleted_at", op="is_null"),
])

Available operators: eq, ne, lt, lte, gt, gte, in, not_in, like, ilike, is_null, is_not_null.

Column names are validated as plain identifiers, and values are passed to the database as bound parameters — never interpolated into SQL. Filter values are therefore safe to build from end-user input.

Batching modes

index() requires exactly one of batch_size or row_ids:

• batch_size=N — dot-search reads all rows and batches embeddings internally in chunks of N.
• row_ids=[...] — the caller decides which rows to embed. dot-search only reads, embeds, and writes those specific rows in a single pass. This lets you drive batching from the outside — useful when you want to control concurrency, retry individual batches, or stream IDs from a queue.

Setup steps (creating columns, saving config) run in both modes — they're idempotent.

Stability

dot-search follows semantic versioning: everything exported from the top-level package is covered, anything underscore-prefixed is internal and may change in any release. Public names are never removed without a deprecation period.

dependencies = ["dot-search>=2.0,<3"]

See docs/VERSIONING.md for the full policy.

Roadmap

• Async engine API
• Reranking stage after fusion
• Incremental re-indexing on row updates

Documentation

Contributing

Contributions are welcome. See CONTRIBUTING.md for the DCO sign-off requirement, the licensing terms that apply to contributions, and how to submit a change.

Licence

Copyright (C) 2026 Kannon For Deep Tech (deepika)

This software is distributed under the GNU Affero General Public License, version 3 or later — see LICENSE.md.

A commercial licence is available for use in proprietary environments. Contact: louis.letarnec@deepika.ai