nseekfs 1.0.1

High-performance exact vector similarity search with Rust backend

NSeekFS

High-Performance Exact Vector Search with Rust Backend

Fast and exact cosine similarity search for Python. Built with Rust for performance, designed for production use.

pip install nseekfs

Quick Start

import nseekfs
import numpy as np

# Create some test vectors
embeddings = np.random.randn(10000, 384).astype(np.float32)
query = np.random.randn(384).astype(np.float32)

# Build index and run a search
index = nseekfs.from_embeddings(embeddings, normalized=True)
results = index.query(query, top_k=10)

print(f"Found {len(results)} results")
print(f"Best match: idx={results[0]['idx']} score={results[0]['score']:.3f}")

Core Features

Exact Search

# Basic query
results = index.query(query, top_k=10)

# Access results
for item in results:
    print(f"Vector {item['idx']}: {item['score']:.6f}")

Batch Queries

queries = np.random.randn(50, 384).astype(np.float32)
batch_results = index.query_batch(queries, top_k=5)
print(f"Processed {len(batch_results)} queries")

Query Options

# Simple query (alias for query with format="simple")
results = index.query_simple(query, top_k=10)

# Detailed query with timing and diagnostics
result = index.query_detailed(query, top_k=10)
print(f"Query took {result.query_time_ms:.2f} ms, top1 idx={result.results[0]['idx']}")

Index Persistence

# Load an existing index
index = nseekfs.from_bin("my_vectors.bin")
print(f"Loaded index: {index.rows} vectors x {index.dims} dims")

Performance Metrics

metrics = index.get_performance_metrics()
print(f"Total queries: {metrics['total_queries']}")
print(f"Average time: {metrics['avg_query_time_ms']:.2f} ms")

Built-in Benchmark

nseekfs.benchmark(vectors=1000, dims=384, queries=100, verbose=True)

API Reference

Index

• from_embeddings(embeddings, normalized=True, verbose=False)
• from_bin(path)

Queries

• query(query_vector, top_k=10)
• query_simple(query_vector, top_k=10)
• query_detailed(query_vector, top_k=10)
• query_batch(queries, top_k=10)

Properties

• index.rows
• index.dims
• index.config

Utilities

• get_performance_metrics()
• benchmark(vectors=..., dims=..., queries=...)

Architecture Highlights

SIMD Optimizations

• AVX2 support for 8x parallelism on compatible CPUs
• Automatic fallback to scalar operations on older hardware
• Runtime detection of CPU capabilities

Memory Management

• Memory mapping for efficient data access
• Thread-local buffers for zero-allocation queries
• Cache-aligned data structures for optimal performance

Batch Processing

• Intelligent batching strategies based on query size
• SIMD vectorization across multiple queries
• Optimized memory access patterns

Installation

# From PyPI
pip install nseekfs

# Verify installation
python -c "import nseekfs; print('NSeekFS installed successfully')"

Technical Details

• Precision: Float32 optimized for standard ML embeddings
• Memory: Efficient memory usage with optimized data structures
• Performance: Rust backend with SIMD optimizations where available
• Compatibility: Python 3.8+ on Windows, macOS, and Linux
• Thread Safety: Safe concurrent access from multiple threads

Performance Tips

# Pre-normalize vectors if using cosine similarity
embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
index = nseekfs.from_embeddings(embeddings, normalized=False)

# Use appropriate data types
embeddings = embeddings.astype(np.float32)

# Choose optimal top_k values
results = index.query(query, top_k=10)  # vs top_k=1000

# Use batch processing for multiple queries
batch_results = index.query_batch(queries, top_k=10)

License

MIT License - see LICENSE file for details.

---

Fast, exact cosine similarity search for Python.

Built with Rust for performance, designed for Python developers.