langchain-infino

LangChain over Infino — vector, full-text (BM25), hybrid, and SQL-native retrieval over one copy of your data on object storage.

Most "vector database" LangChain integrations expose only the vector slice of their engine. Infino keeps your data in Apache Parquet on object storage and runs SQL, BM25, vector, and hybrid (RRF) retrieval over it from a single in-process engine. This package surfaces that whole retrieval surface, not just similarity_search.

What you get

• One store, four retrieval modes — vector, BM25, hybrid (RRF), and raw SQL over the same rows. Nothing to dual-write, no drift between a vector index and a search cluster.
• Storage you already pay for — Parquet on S3 or Azure Blob. No cluster to size, patch, or keep warm; local disk in dev is the same code path.
• Drop-in for existing chains — a standard VectorStore plus retrievers, self-query, and a semantic LLM cache.
• Your embeddings, your choice — Infino never embeds. Bring a LangChain Embeddings object and the integration supplies the vectors.

Installation

pip install langchain-infino

Or with uv:

uv add langchain-infino

Requires Python 3.9+. infino, langchain-core, pyarrow, and numpy are installed as dependencies. Bring your own embeddings provider separately (e.g. pip install langchain-openai).

Quickstart

import infino
from langchain_infino import InfinoVectorStore
from langchain_openai import OpenAIEmbeddings

# A local path or an S3 URI for durable storage; "memory://" is ephemeral.
connection = infino.connect("./data")
embedding = OpenAIEmbeddings()

store = InfinoVectorStore.from_texts(
    ["Infino runs search on object storage.", "One engine for SQL, BM25, and vectors."],
    embedding,
    connection=connection,
    table_name="docs",
)

docs = store.similarity_search("search on S3", k=2)
retriever = store.as_retriever()

Core concepts

• InfinoVectorStore wraps a single Infino table — the text, its embedding, the document id, declared metadata columns, and a JSON catch-all. Use from_texts to create and populate one; construct directly to open an existing table.
• Identity — caller-controlled ids live on Document.id (not in metadata). add_texts is an idempotent upsert: re-adding an id overwrites, omitted ids are generated.
• Metadata, two tiers — keys you name in metadata_columns= become real scalar columns you can filter on; everything else round-trips losslessly through a JSON catch-all but isn't filterable. The schema is fixed at table creation — adding a filterable key means recreating the table.
• Scores — vector distance is smaller is nearer; BM25 and RRF are larger is better. similarity_search_with_relevance_scores normalizes to [0, 1] (higher = better) for cosine, l2, and l2sq.
• Retrievers — as_retriever() (vector), as_bm25_retriever() (lexical), and as_hybrid_retriever() (RRF fusion).
• Dimensions — embeddings must be [16, 4096]-dimensional (engine limit) and match the table's declared dim.

Object storage (S3 / Azure)

The store operates on any infino.Connection, so it runs against local disk or cloud object storage unchanged — the URI and storage_options you pass to infino.connect are the only difference. Keys are the standard object_store config strings (aws_* / azure_*); ambient credentials (IAM role, env vars) need no storage_options at all.

# Amazon S3 (or S3-compatible: set aws_endpoint, aws_allow_http for MinIO/R2).
connection = infino.connect("s3://bucket/prefix", storage_options={
    "aws_access_key_id": "...",
    "aws_secret_access_key": "...",
    "aws_region": "us-east-1",
})

# Azure Blob Storage.
connection = infino.connect("az://container/prefix", storage_options={
    "azure_storage_account_name": "...",
    "azure_storage_account_key": "...",
})

store = InfinoVectorStore.from_texts(
    texts, embedding, connection=connection, table_name="docs", dim=1536,
)

InfinoVectorStore.connect collapses that into one call when you don't need the connection for anything else. It takes the same connection options and reaches a local directory, object storage, or a hosted target the same way:

store = InfinoVectorStore.connect(
    "s3://bucket/prefix", embedding, "docs", dim=1536,
    storage_options={"aws_region": "us-east-1"},
)

It opens an existing table by default; pass create=True to ensure the table exists — created when absent, opened when present, so it is safe on every run — and create_database=True to provision the database first.

The constructors differ only in what they expect of the table: InfinoVectorStore(...) requires it to exist, from_texts(...) requires that it does not, and open_or_create(...) accepts either. Reach for the last when a process attaches to its own table across restarts.

Two connect options worth setting in production:

• validate=True probes the store at connect time, so bad credentials fail there instead of on the first read.
• connection_memory_budget_bytes caps what one connection may hold. An ingest or query that would exceed it raises ConnectionMemoryBudgetError — recoverable, so you can narrow the query, split the ingest, or raise the budget. It subclasses InfinoError, the base for every engine failure. Both are re-exported from langchain_infino, alongside ConflictError for a lost commit race, so handling engine failures needs no second import.

For a hosted Infino target, pass api_key= and provision the database once:

connection = infino.connect("https://...", api_key="...")
connection.create_database()  # no-op against a local or object-store URI

Or in one call:

store = InfinoVectorStore.connect(
    "https://...", embedding, "docs", dim=1536,
    api_key="...", create_database=True, create=True,
)

Adding and managing documents

# Generated ids on the common path; returns them.
ids = store.add_texts(["a new note"], metadatas=[{"source": "inbox"}])

# Caller ids are upserted — re-adding "doc-1" overwrites in place.
store.add_texts(["v2 of the note"], ids=["doc-1"])

# Fetch by id (skips missing, order not guaranteed); delete by id.
store.get_by_ids(["doc-1"])
store.delete(["doc-1"])

# Delete what a filter matches, or — per the VectorStore contract — pass no
# ids to empty the table. An empty list deletes nothing.
store.delete(filter={"source": "inbox"})
store.delete()

# When you need the counts, or a predicate the filter can't express.
stats = store.delete_by_predicate("source = 'inbox' AND year < 2020")
stats.matched, stats.n_tombstoned

delete returns True whenever the delete was issued, including when it matched nothing — deleting ids that aren't there has still succeeded.

Similarity search

store.similarity_search("vector databases", k=4)
store.similarity_search_with_score("vector databases", k=4)       # raw distance
store.similarity_search_with_relevance_scores("vector databases", k=4)  # [0, 1]
store.similarity_search_by_vector(query_vector, k=4)              # query_vector: list[float]

Metadata filtering

Scalar metadata is filterable out of the box — from_texts promotes the keys it finds in metadatas to real columns. Pass the LangChain operator form: equality, $eq / $ne / $gt / $gte / $lt / $lte, $in / $nin, and $and / $or / $not.

store = InfinoVectorStore.from_texts(
    texts, embedding,
    connection=connection, table_name="papers",
    metadatas=[{"category": "ml", "year": 2024} for _ in texts],
)

store.similarity_search("optimizers", k=4, filter={"category": "ml"})
store.similarity_search("optimizers", k=4, filter={"year": {"$gte": 2023}})
store.similarity_search("optimizers", k=4,
                        filter={"$or": [{"category": "ml"}, {"year": {"$lt": 2000}}]})

A key is promoted only if every value it carries is a scalar of one consistent type. Anything nested, mixed-typed, or named like a column the engine reserves (score, _id, _metadata_json) stays in the JSON catch-all: it round-trips with the document, but filtering on it raises rather than scanning, because the engine has no index into serialized JSON.

Promotion happens once, at table creation, from the metadata present then — so declare columns explicitly if later documents will introduce keys you intend to filter on, or if you want a specific type or non-null constraint:

import pyarrow as pa

store = InfinoVectorStore.from_texts(
    texts, embedding,
    connection=connection, table_name="papers",
    metadata_columns=[
        pa.field("category", pa.large_utf8(), nullable=False),
        pa.field("year", pa.int64(), nullable=True),
    ],
    metadatas=[{"category": "ml", "year": 2024} for _ in texts],
)

Opening an existing table needs neither metadata_columns nor dim — both are read back from the table's own schema.

Text-pushdown pre-filter

For a text predicate, push it into the kNN instead of post-filtering the top-k. The engine prunes to rows matching the full-text terms before ranking, so exactly k nearest matching rows come back — no over-fetch, no under-return. filter_mode is "or" (default) or "and"; filter_column defaults to the text column.

store.similarity_search("cancel my plan", k=10, filter_query="subscription billing")

It is reachable from any retriever via search_kwargs:

retriever = store.as_retriever(search_kwargs={"k": 10, "filter_query": "billing"})

filter (structured, post-rank SQL WHERE) and filter_query (text, pre-rank pushdown) are distinct paths and not combinable in one call.

Maximal marginal relevance (MMR)

store.max_marginal_relevance_search("transformers", k=4, fetch_k=20, lambda_mult=0.5)

Infino's vector column isn't projectable and there's no point-lookup, so MMR re-embeds the fetch_k candidates' text to score them against each other.

Hybrid (RRF) retrieval

The default choice when queries mix natural language with exact terms — error codes, SKUs, proper nouns — that pure vector search blurs away. BM25 and vector search are fused by reciprocal-rank fusion in a single call, with no separate reranking round-trip.

retriever = store.as_hybrid_retriever(k=4)
retriever.invoke("neural network training")

BM25 retrieval

Pure lexical ranking over the FTS-indexed text column.

retriever = store.as_bm25_retriever(k=4)              # OR by default
retriever = store.as_bm25_retriever(k=4, mode="and")  # require all terms
retriever.invoke("gradient descent")
retriever.invoke("gradient descent", k=10)            # override per call

A growing table splits across many storage files, and by default each file ranks against its own term statistics — so the same document can score differently depending on which file it landed in. stats="global" ranks against corpus-wide statistics instead, and a large table then behaves exactly like one unified index. It costs one extra document-frequency pass over the files holding your query's terms, so reach for it when ranking quality matters more than the last few milliseconds.

retriever = store.as_bm25_retriever(k=4, stats="global")

Term matching and counting

BM25 ranks and truncates to k. When you want the whole matching set rather than the best few — a filter, an export, an audit — token_search matches terms without ranking, so it takes no k:

store.token_search("gradient descent")               # any term
store.token_search("gradient descent", mode="and")   # both terms

exact_search looks a value up verbatim, with no tokenization at all. It needs an FTS-indexed column; the store indexes the id and text columns:

store.exact_search("doc-1", "doc_id")

count answers how many documents match without materializing any of them, so it stays cheap on a table far larger than memory:

store.count("gradient descent")
store.count("gradient descent", mode="and")

Language and tokenization

Out of the box the text index folds to lowercase ASCII — right for English, but it strips accents and drops non-Latin scripts. If your corpus isn't English, index it with the standard analyzer (UAX #29 word segmentation and full Unicode lowercasing) so terms like café stay searchable.

store = InfinoVectorStore.from_texts(
    texts, embedding,
    connection=connection, table_name="docs", dim=1536,
    analyzer="standard",
)

Pick it at table creation — changing the analyzer later means recreating the table. The id column always keeps the default so get_by_ids matches ids verbatim.

Recall and maintenance

Vector search is approximate: a query probes part of the index, then reranks the survivors against full-precision vectors. How widely it probes and how deep it reranks are engine-decided — calibrated per table from its size and distribution, with no knobs to set per query.

Calibration happens during optimize, which is also what compacts the small immutable files each append leaves behind. Search works without it, but recall and latency both improve once it has run, so run it after a bulk load and periodically under steady ingest:

store.optimize()

Compaction leaves the pre-merge files unreferenced. gc reclaims them, and its grace period spares anything younger so in-flight readers on an older snapshot are not pulled out from under:

report = store.gc(grace_secs=3600)
report.bytes_freed, report.objects_deleted

Self-query

InfinoTranslator plugs into LangChain's SelfQueryRetriever, lowering an LLM's structured query to a SQL WHERE over the declared metadata columns — the full comparison and boolean surface, not a reduced DSL. Pass it as the structured_query_translator (see LangChain's self-query docs for the metadata_field_info setup):

from langchain_infino import InfinoTranslator

retriever = SelfQueryRetriever.from_llm(
    llm,
    store,
    document_contents="research papers",
    metadata_field_info=metadata_field_info,
    structured_query_translator=InfinoTranslator(),
)
retriever.invoke("ML papers since 2023")

SQL-native search

The escape hatch for anything the typed methods don't cover — joins, custom WHERE, or the vector_search / hybrid_search table functions. Project the store's columns (doc_id, page_content, declared metadata, _metadata_json, and optionally score) and the rows map back to Documents.

qv = ",".join(map(str, embedding.embed_query("fox")))
store.search_by_sql(f"""
    SELECT doc_id, page_content, _metadata_json, score
    FROM hybrid_search('docs', 'page_content', 'fox', 'embedding', '{qv}', 10)
    ORDER BY score DESC
""")

Semantic LLM cache

Caches model responses keyed by prompt meaning: a lookup embeds the prompt and returns a hit when a stored prompt for the same model lands within a distance threshold. One small Infino table, no extra infrastructure.

from langchain_core.globals import set_llm_cache
from langchain_infino import InfinoSemanticCache

set_llm_cache(InfinoSemanticCache(connection, embedding, dim=1536))

Async

The async methods (aadd_texts, asimilarity_search, …) are inherited from VectorStore, which offloads the synchronous engine calls to a thread via run_in_executor — the event loop is never blocked.

API reference

• InfinoVectorStore(connection, table_name, embedding, *, dim=None, metric="cosine", text_column="page_content", vector_column="embedding", id_column="doc_id", metadata_columns=None) — opens an existing table.
  • connect(uri, embedding, table_name, *, dim=None, create=False, create_database=False, storage_options=None, cache_dir=None, cache_budget_bytes=None, connection_memory_budget_bytes=None, cold_fetch_mode=None, validate=None, api_key=None, metric="cosine", analyzer=None, text_column=..., vector_column=..., id_column=..., metadata_columns=None) -> InfinoVectorStore — connects and opens (or with create=True, creates) in one call.
  • open_or_create(connection, table_name, embedding, *, dim=None, metric="cosine", analyzer=None, text_column=..., vector_column=..., id_column=..., metadata_columns=None) -> InfinoVectorStore — idempotent: creates the table when absent, opens it when present.
  • from_texts(texts, embedding, metadatas=None, *, connection, table_name="langchain", dim=None, ids=None, metric="cosine", analyzer=None, text_column=..., vector_column=..., id_column=..., metadata_columns=None) -> InfinoVectorStore — creates and populates the table.
  • add_texts(texts, metadatas=None, *, ids=None) -> list[str] — idempotent upsert.
  • similarity_search(query, k=4, filter=None, *, filter_query=None, filter_column=None, filter_mode=None) -> list[Document]
  • similarity_search_with_score(...), similarity_search_by_vector(...)
  • max_marginal_relevance_search(query, k=4, fetch_k=20, lambda_mult=0.5, filter=None, ...)
  • delete(ids=None, *, filter=None) -> bool — by id, by filter, or all of them when ids is None; get_by_ids(ids) -> list[Document]
  • delete_by_predicate(predicate) -> MutationStats — raw SQL, with counts
  • token_search(query, *, column=None, mode=None) -> list[Document] — unranked term match, no k.
  • exact_search(value, column) -> list[Document] — verbatim key lookup.
  • count(query, *, column=None, mode=None) -> int — counts in the engine.
  • optimize(*, max_memory_mb=None, min_fill_percent=None, target_superfile_size_mb=None, stale_seal_timeout_ms=None) -> None
  • gc(grace_secs) -> GcReport, schema() -> pyarrow.Schema, drop(*, purge=True)
  • search_by_sql(sql) -> list[Document]
  • as_retriever(...), as_hybrid_retriever(k=4), as_bm25_retriever(k=4, mode=None, *, stats=None)
  • connection, table, table_name, metric, dim, metadata_columns — accessors, including for engine calls the store doesn't wrap.
• InfinoHybridRetriever, InfinoBM25Retriever — BaseRetrievers wrapping a store.
• InfinoTranslator — StructuredQuery → SQL filter, for SelfQueryRetriever.
• InfinoSemanticCache(connection, embedding, *, dim=None, table_name="langchain_llm_cache", score_threshold=0.05)
• InfinoError and its recoverable subclasses ConflictError (a concurrent writer won the commit race — reissue) and ConnectionMemoryBudgetError (the request exceeded connection_memory_budget_bytes — narrow it, split the ingest, or raise the budget), plus MutationStats and GcReport.

metric is "cosine" (default), "l2sq" / "l2", or "negdot" / "dot"; dim and metadata_columns are inferred when omitted — from the embedding and metadatas when creating a table, from the table's own schema when opening one. analyzer is "ascii_lower" (default) or "standard"; stats is "per_superfile" (default) or "global"; cold_fetch_mode is "hybrid_with_prefetch", "range_only", or "lazy_foreground_with_background_fill". See Infino for engine internals.

Development

make install      # pip install -e ".[test,lint]"
make unit         # unit tests (no engine)
make integration  # integration + compliance tests (real Infino on a temp dir)
make lint type    # ruff + mypy
make build        # build sdist + wheel into dist/
make smoke        # build the wheel, install it in a clean venv, run the smoke test
make clean        # remove build artifacts and caches

License

Apache-2.0.