vector_cache_lmdb

vector_cache_lmdb is a Rust library with Python bindings for storing and retrieving Vec<f32> embeddings by text key.

It uses LMDB via heed for durable mmap-backed storage with many concurrent readers and safely serialized writers across multiple processes.

Installation

pip install vector-cache-lmdb

Python usage

from vector_cache_lmdb import VectorCache

cache = VectorCache("/var/opt/cache.bin", max_items=1_000_000)

cache.put("this is a test string", [0.1, 0.5, 1.1])
print(cache.get("this is a test string"))

cache.put_multi(
    ["first string", "second string"],
    [[0.1, 0.5, 1.1], [2.0, 3.0, 4.0]],
)

print(cache.get_multi(["first string", "second string", "missing"]))
print(cache.delete("this is a test string"))
print(cache.delete_multi(["first string", "second string", "missing"]))

cache.reset()

Capacity can be configured in one of two mutually exclusive ways:

# Count-based capacity
cache = VectorCache("/var/opt/cache.bin", max_items=5_000_000)

# Byte-based capacity (vector payload bytes), fixed dimension required
cache = VectorCache("/var/opt/cache.bin", max_gb=2.0)

Passing both max_items and max_gb raises ValueError.

OpenAI Embeddings Helper

Use get_embeddings_with_cache with make_openai_embed_fn to keep ordering exact while filling cache misses:

from openai import OpenAI

from vector_cache_lmdb import (
    VectorCache,
    get_embeddings_with_cache,
    make_openai_embed_fn,
)

client = OpenAI()
cache = VectorCache("/var/opt/cache.bin", max_items=1_000_000)
embed_fn = make_openai_embed_fn("text-embedding-3-small", client=client)

texts = ["first string", "second string", "first string"]
embeddings = get_embeddings_with_cache(
    texts=texts,
    cache=cache,
    embed_fn=embed_fn,
)

make_openai_embed_fn loads .env automatically when client is omitted.

Sentence-Transformers Embeddings Helper

from sentence_transformers import SentenceTransformer

from vector_cache_lmdb import (
    VectorCache,
    get_embeddings_with_cache,
    make_sentence_transformers_embed_fn,
)

encoder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
cache = VectorCache("/var/opt/cache.bin", max_items=1_000_000)
embed_fn = make_sentence_transformers_embed_fn(encoder)

texts = ["first string", "second string", "first string"]
embeddings = get_embeddings_with_cache(
    texts=texts,
    cache=cache,
    embed_fn=embed_fn,
)

Generic Custom Provider

from vector_cache_lmdb import get_embeddings_with_cache

def my_embed_fn(texts: list[str]) -> list[list[float]]:
    return my_provider.embed(texts)  # must return list[list[float]]

embeddings = get_embeddings_with_cache(
    texts=texts,
    cache=cache,
    embed_fn=my_embed_fn,
)

Notes

• The path argument is an LMDB environment directory (not a single data file).
• Hashing uses BLAKE3 for very fast fixed-size text keys.
• get_multi preserves input order and returns None for missing entries.
• Capacity is enforced with an on-disk LRU index shared across processes.
• Eviction runs on writes:
  • max_items: when len() > max_items
  • max_gb: when bytes_len() > max_gb budget (vector payload bytes)
• In max_gb mode, vector dimension is detected and must remain fixed.
• For correctness, use a local filesystem path. LMDB locking semantics are not guaranteed on remote/network filesystems (for example SSHFS/NFS mounts).