embeddb 0.2.0

Tiny semantic search DB in one file. Zero config. Instant prototyping.

EmbedDB

Tiny semantic search DB in one file. Zero config. Instant prototyping.

EmbedDB is an ultra-lightweight vector database designed for rapid prototyping of semantic search and RAG applications. The entire implementation is contained in a single Python file with minimal dependencies.

Why EmbedDB?

• ⚡ 2-Minute Integration: From pip install to working code in under 2 minutes
• 📄 Single File: Entire implementation in one readable Python file (<500 lines)
• 🔍 Simple API: Just 7 intuitive commands to learn
• 🛠️ Zero Config: No accounts, API keys, or complex setup
• 💾 JSON Persistence: Simple file-based storage with human-readable format
• 🧩 Extensible: Easy to modify or extend for your specific needs
• 🤖 Optional Embeddings: Built-in text embedding support (when you need it)

Installation

Basic installation (vector storage only):

pip install embeddb

With built-in embedding support:

pip install embeddb[embeddings]

That's it. No compilation. No complex dependencies.

Quickstart

Just 7 Commands. 2 Minutes to Get Started.

EmbedDB is designed for simplicity. The entire API consists of just 7 intuitive commands:

db = EmbedDB(dimension=384)       # Create a database
db.add(id, vector, metadata)      # Add a vector
db.search(vector, top_k=5)        # Find similar vectors
db.get(id)                        # Retrieve a vector
db.delete(id)                     # Remove a vector
db.save(filepath)                 # Persist to disk
db.load(filepath)                 # Load from disk

That's it. No complex configuration. No steep learning curve. Just install and start building.

How EmbedDB Differs from Other Vector Databases

Feature	EmbedDB	FAISS	Pinecone/Weaviate/etc.
Installation	pip install embeddb	Requires C++ compilation	Requires accounts, API keys
Setup time	< 2 minutes	10-30 minutes	30+ minutes
Dependencies	Just NumPy	BLAS, C++ toolchain	External services
Learning curve	Minimal (7 commands)	Steep (many index types)	Moderate to steep
Storage	Simple JSON files	Custom binary format	Cloud-hosted
Best for	Rapid prototyping	Production performance	Large-scale production
Metadata	Built-in JSON	Requires separate handling	Built-in
Coding style	Pure Python	C++ with Python bindings	Client/server

Performance Benchmarks

While EmbedDB prioritizes ease of use over raw performance, it's still remarkably efficient for prototyping and small to medium-sized applications.

Measured Performance

Real benchmarks show EmbedDB's efficient scaling characteristics:

1. Adding Vectors: Consistent performance regardless of database size (~0.017ms per vector)
2. Search Performance: Linear scaling from 1ms at 100 vectors to ~110ms at 10,000 vectors
3. Memory Efficiency: Linear memory growth with database size (2MB at 1,000 vectors, 130MB at 10,000 vectors)

Benchmarking Results

Database Size	Add (ms/vector)	Search (ms)	Memory (MB)
100 vectors	0.0188	1.3	1.5
1,000 vectors	0.0166	11.5	15.4
10,000 vectors	0.0182	113.0	130.5

Benchmark details: Tests performed on a modern system using 384-dimension vectors with small metadata.

Performance Recommendations

• <1,000 vectors: Blazing fast with minimal memory footprint
• 1,000-10,000 vectors: Excellent performance for most prototyping needs
• 10,000-100,000 vectors: Suitable for medium-sized applications
• >100,000 vectors: Consider more specialized solutions like FAISS for production

Working with Vectors

from embeddb import EmbedDB

# Create a vector database
db = EmbedDB(dimension=384)  # Set to your embedding dimension

# Add vectors with metadata
db.add("doc1", [0.1, 0.2, ...], {"text": "EmbedDB is easy to use"})
db.add("doc2", [0.3, 0.1, ...], {"text": "Vector databases for everyone"})

# Search for similar vectors
results = db.search([0.2, 0.3, ...], top_k=2)
for result in results:
    print(f"Match: {result['metadata']['text']} (Score: {result['similarity']})")

# Save for later
db.save("my_database.json")

# Load back when needed
db = EmbedDB.load("my_database.json")

Working with Text (with built-in embeddings)

from embeddb import EmbedDB

# Create a database with built-in embedding support
db = EmbedDB()

# Add documents with automatic embedding
db.add_text("doc1", "EmbedDB makes vector search easy")
db.add_text("doc2", "Semantic search finds related content")

# Search with a text query
results = db.search_text("How do I find similar documents?")

for result in results:
    print(f"Match: {result['metadata']['text']}")

Complete RAG Example

Here's a complete Retrieval-Augmented Generation implementation in under 30 lines:

from embeddb import EmbedDB

# Initialize EmbedDB with built-in embedding support
db = EmbedDB()  # Uses all-MiniLM-L6-v2 under the hood

# Add some documents (automatic embedding)
documents = [
    "EmbedDB is a lightweight vector database for prototyping.",
    "Vector databases store embeddings for semantic search.",
    "RAG systems enhance LLMs with relevant document retrieval."
]

# Add documents to EmbedDB
for i, doc in enumerate(documents):
    db.add_text(f"doc{i}", doc)

# Perform a search with text query
query = "How can I use vector search in my project?"
results = db.search_text(query, top_k=2)

# Display results
print(f"Query: {query}")
print("Relevant documents:")
for i, result in enumerate(results):
    print(f"{i+1}. {result['metadata']['text']} (Score: {result['similarity']:.4f})")

API Reference

Core Vector Database Functions:

# Create a database
db = EmbedDB(dimension=None)  # Dimension optional, set on first add

# Add a vector
db.add(id, vector, metadata)  # metadata can be any JSON-serializable object

# Search for similar vectors
results = db.search(query_vector, top_k=5)  # Returns list of {id, similarity, metadata}

# Get a vector by ID
vector, metadata = db.get(id)

# Delete a vector
db.delete(id)

# Save database to file
db.save(filepath)

# Load database from file
db = EmbedDB.load(filepath)

# Count vectors
count = db.count()

Text Embedding Functions (with embeddb[embeddings]):

# Generate embeddings for text
vector = db.embed_text(text)

# Add text directly (auto-generates embedding)
db.add_text(id, text, metadata=None)

# Search using text query
results = db.search_text(query_text, top_k=5)

When to Use EmbedDB

✅ Perfect for:

• Rapid prototyping of semantic search applications
• Learning and education about vector databases
• Small to medium-sized RAG implementations
• Projects where simplicity is more important than scale
• Quick demos and proofs of concept

❌ Not designed for:

• Production systems with millions of vectors
• High-throughput applications
• Advanced filtering and hybrid search
• Multi-user environments requiring access control

How It Works

EmbedDB uses in-memory storage with cosine similarity for vector search. Vectors are automatically normalized on insertion for optimal similarity calculation. Persistence is handled through simple JSON serialization.

The optional text embedding functionality uses the all-MiniLM-L6-v2 model from sentence-transformers, which provides an excellent balance between size, speed, and quality for most prototyping needs.

The database has virtually no learning curve - if you know how to call functions in Python, you know how to use EmbedDB.

Contributing

Contributions are welcome! The goal of this project is to remain extremely simple while being useful for developers. Before adding features, please consider whether they align with the project's minimalist philosophy.

License

MIT License

---

Made for developers who want to go from idea to prototype in minutes, not days.