llmemory 0.4.0

High-performance document memory system with vector search capabilities for Python applications

llmemory

A high-performance document memory system with vector search capabilities for Python applications.

Overview

llmemory provides intelligent document processing with:

• Complete Document Management API – List, retrieve, search, and manage documents without direct database access
• State-of-the-art retrieval using PostgreSQL with pgvector, hybrid BM25, multi-query expansion, and reranking
• Multi-language support with automatic language detection and normalization
• Hierarchical chunking & summaries with document-type specific configurations and optional auto-summaries
• Production-ready monitoring with Prometheus metrics and searchable diagnostics

What's New

• 🔁 Multi-query expansion – Generate semantic + keyword variants automatically and fuse results with reciprocal rank fusion
• 🎯 Configurable reranking – Plug in OpenAI or cross-encoder rerankers (or use built-in heuristics) for higher precision on the final hit list
• 🧭 Query routing – Automatic answerability detection routes queries to best retrieval strategy
• 🎨 Contextual retrieval – Anthropic-style chunk enrichment with document context for improved semantic matching
• ⚙️ HNSW presets – Choose fast, balanced, or accurate profiles to tune pgvector index parameters and query-time ef_search
• 📝 Chunk summaries – Capture short, metadata-aware synopses during ingestion and surface them with every search hit
• 📈 Richer diagnostics – Search history now records query variants, latency breakdowns, rerank status, and summary usage for easy tuning

Why llmemory?

Building applications with document search capabilities requires solving complex problems:

• Vector embeddings for semantic understanding
• Efficient chunking that preserves context
• Hybrid search combining vectors and full-text
• Multi-tenant isolation for SaaS applications
• Performance optimization for large document sets

llmemory provides a production-ready solution for these challenges.

Key Features

• 🚀 Fast Search – HNSW indexes for sub-100 ms vector searches, with multi-query expansion and optional cross-encoder reranking for harder queries
• 🌍 Multi-language – Automatic detection and processing for 14+ languages
• 📊 Smart Chunking – Document-type aware chunking with contextual enrichment, optional inline summaries, and hierarchical parent context
• 🔍 Hybrid Search – Combines vector and text search with reciprocal rank fusion, query routing, and rerank scores
• 📈 Observable – Built-in Prometheus metrics and detailed search diagnostics
• 🏢 Multi-tenant – Owner-based isolation for SaaS applications
• 🔌 Flexible Embeddings – Support for OpenAI and local embedding models

Quick Start

from llmemory import LLMemory, DocumentType, SearchType

# Initialize
memory = LLMemory(
    connection_string="postgresql://localhost/mydb",
    openai_api_key="sk-..."
)
await memory.initialize()

# Add a document - returns detailed results
result = await memory.add_document(
    owner_id="workspace-123",
    id_at_origin="user-456",
    document_name="project-report.pdf",
    document_type=DocumentType.REPORT,
    content="Your document content here..."
)
print(f"Created {result.chunks_created} chunks in {result.processing_time_ms}ms")

# List documents with filtering
docs = await memory.list_documents(
    owner_id="workspace-123",
    document_type=DocumentType.REPORT,
    metadata_filter={"status": "active"}
)

# Search with document metadata
results = await memory.search_with_documents(
    owner_id="workspace-123",
    query_text="project timeline",
    search_type=SearchType.HYBRID
)
for result in results.results:
    print(f"Found in: {result.document_name} - {result.content[:100]}...")

# Get statistics
stats = await memory.get_statistics("workspace-123")
print(f"Total: {stats.document_count} docs, {stats.chunk_count} chunks")

Installation

# Install using uv (recommended)
uv add llmemory

# Or using pip
pip install llmemory

Or with optional dependencies:

# Using uv
uv add "llmemory[monitoring]"  # For Prometheus metrics
uv add "llmemory[cache]"       # For Redis caching
uv add "llmemory[local]"       # For local embeddings
uv add "llmemory[reranker-local]"  # For local cross-encoder reranking
uv add "llmemory[bench]"       # For BEIR benchmarking harness

# Using pip
pip install "llmemory[monitoring]"  # For Prometheus metrics
pip install "llmemory[cache]"       # For Redis caching
pip install "llmemory[local]"       # For local embeddings
pip install "llmemory[reranker-local]"  # For cross-encoder reranking support
pip install "llmemory[bench]"       # For benchmarking harness

Documentation

• 📖 Installation Guide - Detailed setup instructions
• 🚀 Quick Start - Get running in 5 minutes
• 🎯 Usage Patterns - Standalone vs shared pool patterns
• 📚 API Reference - Complete API documentation
• 🔧 Integration Guide - Framework integration patterns
• 🗄️ Migration Guide - How migrations work in each pattern
• 📊 Monitoring Guide - Production monitoring setup
• 💡 Examples - Working examples for common use cases
• 🧪 bench/beir_runner.py - BEIR benchmarking harness (requires llmemory[bench])

Performance

• Search latency: < 100ms (p95) with proper indexing
• Throughput: 1000+ searches/second with caching
• Document processing: Handles documents up to 1MB efficiently
• Multi-language: Processes 14+ languages with automatic detection

Requirements

• PostgreSQL 14+ with pgvector extension
• Python 3.10+
• OpenAI API key (or local embedding models)

License

MIT License - see LICENSE file for details.