vector-embed-cache 0.1.0

Cache embedding vectors locally with smart fallback to remote compute

vector-embed-cache

A Python client library that caches embedding vectors locally with smart fallback to a hosted backend. Eliminates API costs and latency through local caching while maintaining reliability.

Features

🚀 Local SQLite cache for zero-latency lookups - Cache all embeddings in a local database for instant retrieval

🔄 Smart fallback chain: local model → remote backend - Automatically falls back to remote if local fails

💰 Zero API costs using nomic-embed-text-v1.5 - Run embeddings locally with open-source models

📊 Cache statistics tracking hits/misses - Monitor cache performance and efficiency

🎯 Simple API with advanced options for power users - Start simple, scale to complex use cases

Why vector-embed-cache?

vs RedisVL / Remote Caches

• No Redis setup required - Just SQLite, no external dependencies
• Works offline - Local model runs without internet
• Zero network latency - Cache lookups are disk reads, not network calls
• Computes embeddings - Built-in model support, not just caching

vs GPTCache / Semantic Caches

• Exact matching - Deterministic cache keys, no similarity search needed
• Simpler setup - Single SQLite file vs vector stores + embedding + eviction
• Text normalization - "Hello" and "hello" share the same cache entry
• Embedding-focused - Designed for vectors, not LLM response caching

vs Generic Caches (diskcache, cachew)

• Embedding-aware - Automatic normalization and provider fallback
• Stats tracking - Built-in monitoring for cache hits/misses
• Smart fallback - Local model → remote backend → error
• Zero config - Works out of the box with sensible defaults

When to use vector-embed-cache

✅ Prototyping with embeddings locally ✅ Reducing API costs for embedding computation ✅ Offline or air-gapped environments ✅ Projects that need fallback to remote compute ✅ Simple single-file cache without infrastructure

When to use alternatives

• Need semantic similarity matching → GPTCache
• Already have Redis infrastructure → RedisVL
• Caching non-embedding data → diskcache
• Need distributed caching → Redis/Memcached

Installation

Basic installation:

pip install vector-embed-cache

With local model support:

pip install vector-embed-cache[local]

With OpenAI support:

pip install vector-embed-cache[openai]

With both local and OpenAI support:

pip install vector-embed-cache[local,openai]

Development installation:

pip install -e .[dev,local]

Usage with OpenAI

Set your API key:

export OPENAI_API_KEY=your-key-here

Use OpenAI embeddings:

from vector_embed_cache import EmbeddingCache

# Create cache with OpenAI model (note the "openai:" prefix)
cache = EmbeddingCache(model="openai:text-embedding-3-small")

# Use it like any other provider
vector = cache.embed("hello world")
print(vector.shape)  # (1536,)

# Subsequent calls hit the cache
vector2 = cache.embed("hello world")
print(cache.stats)
# {'hits': 1, 'misses': 1, 'remote_hits': 0}

The OpenAI provider:

• Uses the OpenAI API with automatic retries
• Caches embeddings locally just like other providers
• Requires OPENAI_API_KEY environment variable
• Supports batch embedding (up to 2048 texts per request)
• Returns 1536-dimensional embeddings for text-embedding-3-small

Model Comparison

embedding-cache supports multiple embedding models. Here's how they compare:

Model	Dimensions	Provider	Cost	Speed	Use Case
nomic-ai/nomic-embed-text-v1.5	768	Local	$0	Fast (cached)	General purpose, offline
nomic-ai/nomic-embed-text-v2-moe	768	Local	$0	Medium (MoE)	Higher quality, offline
openai:text-embedding-3-small	1536	API	$0.0001/1K tokens	Fast (API)	Highest quality, online

Switching Models

from vector_embed_cache import EmbeddingCache

# Use v1.5 (default, fast and reliable)
cache_v15 = EmbeddingCache(model="nomic-ai/nomic-embed-text-v1.5")

# Use v2-moe (newer, potentially higher quality)
cache_v2 = EmbeddingCache(model="nomic-ai/nomic-embed-text-v2-moe")

# Use OpenAI (highest quality, requires API key)
cache_openai = EmbeddingCache(model="openai:text-embedding-3-small")

All models benefit from the same caching layer, so repeated queries are instant regardless of which model you choose.

Quick Start

Simple Function API

The simplest way to use embedding-cache is with the embed() function:

from vector_embed_cache import embed

# Single string
vector = embed("hello world")
print(vector.shape)  # (768,)

# Multiple strings
vectors = embed(["hello", "world"])
print(len(vectors))  # 2
print(vectors[0].shape)  # (768,)

Advanced Class-Based API

For more control over configuration, use the EmbeddingCache class:

from vector_embed_cache import EmbeddingCache

# Create cache with custom settings
cache = EmbeddingCache(
    cache_dir="~/.my-cache",
    model="nomic-ai/nomic-embed-text-v1.5",
    remote_url="https://api.example.com/embed",
    fallback_providers=["local", "remote"],
    timeout=10.0
)

# Use the cache
vector = cache.embed("hello world")

# Access statistics
print(cache.stats)
# {'hits': 0, 'misses': 1, 'remote_hits': 0}

# Subsequent calls will hit the cache
vector2 = cache.embed("hello world")
print(cache.stats)
# {'hits': 1, 'misses': 1, 'remote_hits': 0}

Configuration

Environment Variables

• EMBEDDING_CACHE_DIR: Set custom cache directory location

  export EMBEDDING_CACHE_DIR=/path/to/cache
  

• EMBEDDING_CACHE_URL: Set default remote backend URL

  export EMBEDDING_CACHE_URL=https://api.example.com/embed
  

Cache Location

By default, embeddings are cached in:

• Linux/macOS: ~/.cache/embedding-cache/
• Windows: C:\Users\<username>\.cache\embedding-cache\

Override with EMBEDDING_CACHE_DIR environment variable or the cache_dir parameter.

Override Methods

You can override cache behavior using the EmbeddingCache constructor:

cache = EmbeddingCache(
    cache_dir="/custom/path",           # Custom cache location
    model="nomic-ai/nomic-embed-text-v1.5",  # Custom model
    remote_url="https://api.example.com",    # Remote backend
    fallback_providers=["local", "remote"],  # Provider order
    timeout=5.0                          # Remote timeout (seconds)
)

Architecture

Caching Flow

1. Input Normalization: Text is normalized (whitespace, lowercasing) to maximize cache hits
2. Cache Key Generation: SHA-256 hash generated from (normalized_text + model_name)
3. Cache Lookup: Check SQLite database for existing embedding
4. Cache Hit: Return cached embedding immediately (zero latency)
5. Cache Miss: Generate embedding using fallback chain
6. Cache Store: Save new embedding to SQLite for future lookups

Models

The default model is nomic-embed-text-v1.5 from nomic-ai:

• Dimensions: 768
• Max Tokens: 8192
• Performance: MTEB score of 62.39
• License: Open-source (Apache 2.0)
• Requirements: sentence-transformers, torch, einops

Install with: pip install vector-embed-cache[local]

Testing

Run unit tests:

pytest tests/ -m "not integration"

Run integration tests:

pytest tests/ -m integration

Run all tests:

pytest tests/

Run with coverage:

pytest tests/ --cov=vector_embed_cache --cov-report=html

Development

Clone the repository:

git clone https://github.com/signalnine/embedding-cache.git
cd embedding-cache

Install in development mode:

pip install -e .[dev,local]

Run tests:

pytest

Run tests with coverage:

pytest --cov=vector_embed_cache --cov-report=html
open htmlcov/index.html  # macOS
# or: xdg-open htmlcov/index.html  # Linux
# or: start htmlcov/index.html     # Windows

Error Handling

The library provides clear error messages for common issues:

• ValueError: Empty string or None input

  embed("")  # ValueError: Text cannot be empty string
  embed(None)  # ValueError: Text cannot be None
  

• TypeError: Invalid input type

  embed(123)  # TypeError: Text must be string or list of strings, got int
  embed(["hello", 123])  # TypeError: All items in list must be strings, got int
  

• RuntimeError: All providers fail

  # When no providers are available:
  # RuntimeError: All embedding providers failed. Install sentence-transformers...
  

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Here's how to contribute:

1. Fork the repository on GitHub
2. Create a feature branch: git checkout -b feature/my-new-feature
3. Make your changes and add tests
4. Run the test suite: pytest tests/
5. Submit a pull request with a clear description of your changes

Please ensure your code:

• Passes all existing tests
• Includes tests for new functionality
• Follows the existing code style
• Includes docstrings for new functions/classes

Roadmap

Future features under consideration:

• Support for additional embedding models (OpenAI, Cohere, etc.)
• Batch embedding optimization for large datasets
• Cache compression to reduce disk usage
• Async API support for concurrent embedding requests
• Cache export/import for sharing between systems