endee 1.0.0

Endee is the Next-Generation Vector Database for Scalable, High-Performance AI

Endee - High-Performance Vector Database

Endee is a high-performance vector database designed for speed and efficiency. It enables rapid Approximate Nearest Neighbor (ANN) searches for applications requiring robust vector search capabilities with advanced filtering, metadata support, and hybrid search combining dense and sparse vectors.

Key Features

• Fast ANN Searches: Efficient similarity searches on vector data using HNSW algorithm
• Hybrid Search: Combine dense and sparse vectors for powerful semantic + keyword search
• Multiple Distance Metrics: Support for cosine, L2, and inner product distance metrics
• Metadata Support: Attach and search with metadata and filters
• Advanced Filtering: Powerful query filtering with operators like $eq, $in, and $range
• High Performance: Optimized for speed and efficiency
• Scalable: Handle millions of vectors with ease
• Configurable Precision: Multiple precision levels for memory/accuracy tradeoffs

Installation

pip install endee

Quick Start

from endee import Endee, Precision

# Initialize client with your API token
client = Endee(token="your-token-here")
# for no auth development use the below initialization
# client = Endee()

# List existing collections
collections = client.list_collections()

# Create a new collection
client.create_collection(
    name="my_vectors",
    dimension=1536,              # Your vector dimension
    space_type="cosine",         # Distance metric (cosine, l2, ip)
    precision=Precision.INT8    # Use precision enum for type safety
)

# Get collection reference
collection = client.get_collection(name="my_vectors")

# Insert objects
collection.upsert([
    {
        "id": "doc1",
        "vector": [0.1, 0.2, 0.3, ...],  # Your vector data
        "meta": {"text": "Example document", "category": "reference"},
        "filter": {"category": "reference", "tags": "important"}
    }
])

# Query similar objects with filtering
results = collection.query(
    vector=[0.2, 0.3, 0.4, ...],  # Query vector
    top_k=10,
    filter=[{"category": {"$eq": "reference"}}]  # Structured filter
)

# Process results
for item in results:
    print(f"ID: {item['id']}, Similarity: {item['similarity']}")
    print(f"Metadata: {item['meta']}")

Basic Usage

To interact with the Endee platform, you'll need to authenticate using an API token. This token is used to securely identify your workspace and authorize all actions — including collection creation, object upserts, and queries.

Not using a token at any development stage will result in open APIs and data.

🔐 Generate Your API Token

• Each token is tied to your workspace and should be kept private
• Once you have your token, you're ready to initialize the client and begin using the SDK

Initializing the Client

The Endee client acts as the main interface for all operations — such as creating collections, upserting objects, and running similarity queries. You can initialize the client in just a few lines:

from endee import Endee

# Initialize with your API token
client = Endee(token="your-token-here")

Setting Up Your Domain

The Endee client allows for the setting of custom domain URL and port change (default port 8080).

from endee import Endee

# Initialize with your API token
client = Endee(token="your-token-here")

client.set_base_url('http://0.0.0.0:8081/api/v1')

Collection Methods

The Endee SDK provides collection methods for managing your vector collections. Collections are the primary way to organize and interact with your vector data.

Listing All Collections

The client.list_collections() method returns a list of all the collections currently available in your workspace.

from endee import Endee

client = Endee(token="your-token-here")

# List all collections in your workspace
collections = client.list_collections()

Create a Collection

The client.create_collection() method initializes a new vector collection with customizable parameters such as dimensionality, distance metric, graph construction settings, and precision level.

from endee import Endee, Precision

client = Endee(token="your-token-here")

# Create a collection with custom parameters
client.create_collection(
    name="my_collection",
    dimension=768,
    space_type="cosine",
    M=16,                        # Graph connectivity parameter (default = 16)
    ef_con=128,                  # Construction-time parameter (default = 128)
    precision=Precision.INT8,   # Use Precision enum (recommended)
)

Parameters:

• name: Unique name for your collection (alphanumeric + underscores, max 48 chars)
• dimension: Vector dimensionality (must match your embedding model's output, min 2, max 8000)
• space_type: Distance metric - "cosine", "l2", or "ip" (inner product)
• M: HNSW graph connectivity parameter - higher values increase recall but use more memory (default: 16)
• ef_con: HNSW construction parameter - higher values improve index quality but slow down indexing (default: 128)
• precision: Vector precision level using Precision enum - Precision.INT16, Precision.BINARY2, Precision.INT8 (default), Precision.FLOAT16, or Precision.FLOAT32
• version: Optional version parameter for collection versioning
• sparse_model: Optional parameter for enabling sparse vectors ("default" or "endee_bm25"). By default it is disabled.

Precision Levels:

The precision parameter controls how vectors are stored internally, affecting memory usage and search accuracy. Use the Precision enum for type safety and IDE autocomplete:

from endee import Precision

# Available precision levels
Precision.FLOAT32  # 32-bit floating point
Precision.FLOAT16  # 16-bit floating point
Precision.INT16   # 16-bit integer quantization (default)
Precision.INT8    # 8-bit integer quantization
Precision.BINARY2  # 1-bit binary quantization

Precision	Quantization	Data Type	Memory Usage	Accuracy	Use Case
Precision.FLOAT32	32-bit	FP32	Highest	Maximum	When accuracy is absolutely critical
Precision.FLOAT16	16-bit	FP16	~50% less	Very good	Good accuracy with half precision
Precision.INT16	16-bit	INT16	~50% less	Very good	Default - Integer quantization with good accuracy
Precision.INT8	8-bit	INT8	~75% less	Good	great for most use cases
Precision.BINARY2	1-bit	Binary	~96.9% less	Lower	Extreme compression for large-scale similarity search

Choosing the Right Precision:

• Precision.INT8(Default): - provides good accuracy with significant memory savings using 8-bit integer quantization
• Precision.INT16/ Precision.FLOAT16: Better accuracy with moderate memory savings (16-bit precision)
• Precision.FLOAT32: Maximum accuracy using full 32-bit floating point (highest memory usage)
• Precision.BINARY2: Extreme compression for very large-scale deployments where memory is critical and lower accuracy is tolerable

Example with different precision levels:

from endee import Endee, Precision

client = Endee(token="your-token-here")

# High accuracy collection
client.create_collection(
    name="high_accuracy",
    dimension=768,
    space_type="cosine",
    precision=Precision.FLOAT32
)

# Balanced collection
client.create_collection(
    name="balanced",
    dimension=768,
    space_type="cosine",
    precision=Precision.INT8
)

Get a Collection

The client.get_collection() method retrieves a reference to an existing collection. This is required before performing object operations like upsert, query, or delete.

from endee import Endee

client = Endee(token="your-token-here")

# Get reference to an existing collection
collection = client.get_collection(name="my_collection")

# Now you can perform operations on the collection
print(collection.describe())

Parameters:

• name: Name of the collection to retrieve

Returns: A Collection instance configured with server parameters. Results are cached using LRU cache (max 10 entries) for performance.

Delete a Collection

Collection deletion permanently removes the entire collection and all objects associated with it.

from endee import Endee

client = Endee(token="your-token-here")

# Delete an entire collection
client.delete_collection("my_collection")

⚠️ Caution: Deletion operations are irreversible. Ensure you have the correct collection name before performing deletion.

Object Methods

Once you have a collection reference via client.get_collection(), you can manage individual objects (vectors with metadata) within that collection.

Add Objects

The collection.upsert() method is used to add or update objects in an existing collection. Each object contains a unique identifier, the vector data, optional metadata, and optional filter fields.

from endee import Endee

client = Endee(token="your-token-here")

# Get collection reference
collection = client.get_collection(name="my_collection")

# Insert multiple objects in a batch
collection.upsert([
    {
        "id": "obj1",
        "vector": [...],  # Your vector
        "meta": {"title": "First document"},
        "filter": {"tags": "important"}
    },
    {
        "id": "obj2",
        "vector": [...],  # Another vector
        "meta": {"title": "Second document"},
        "filter": {"visibility": "public", "tags": "important"}
    }
])

Object Fields:

• id: Unique identifier for the object (required)
• vector: Array of floats representing the embedding (required)
• meta: Arbitrary metadata object for storing additional information (optional)
• filter: Key-value pairs for structured filtering during queries (optional)

Note: Maximum batch size is 10000 objects per upsert call.

Query Objects

The collection.query() method performs a similarity search in the collection using a given query vector.

from endee import Endee

client = Endee(token="your-token-here")

collection = client.get_collection(name="my_collection")

# Query with custom parameters
results = collection.query(
    vector=[...],         # Query vector
    top_k=5,              # Number of results to return (max 4096)
    ef=128,               # Runtime parameter for search quality (max 1024)
    include_vectors=True  # Include vector data in results
)

# Process results
for item in results:
    print(f"ID: {item['id']}, Similarity: {item['similarity']}")
    print(f"Metadata: {item['meta']}")

Query Parameters:

• vector: Query vector (must match collection dimension)
• top_k: Number of nearest neighbors to return
• ef: Runtime search parameter - higher values improve recall but increase latency
• include_vectors: Whether to return the actual vector data in results (default: False)
• filter: Optional filter criteria (array of filter objects)
• sparse_indices: Sparse vector indices for hybrid search (default: None)
• sparse_values: Sparse vector values for hybrid search (default: None)
• prefilter_cardinality_threshold: Controls when the search strategy switches from HNSW filtered search to brute-force prefiltering. See Filter Tuning for details.
• filter_boost_percentage: Expands the internal HNSW candidate pool by this percentage when a filter is active. See Filter Tuning for details.

Get Object by ID

The collection.get_object() method retrieves a specific object from the collection by its unique identifier.

from endee import Endee

client = Endee(token="your-token-here")
collection = client.get_collection(name="my_collection")

# Retrieve a specific object by its ID
obj = collection.get_object("obj1")

# The returned object contains:
# - id: Object identifier
# - meta: Metadata dictionary
# - filter: Filter fields dictionary
# - norm: Vector norm value
# - vector: Vector data array
# - sparse_indices: Sparse vector indices (hybrid collections only)
# - sparse_values: Sparse vector values (hybrid collections only)

Delete Object by ID

The collection.delete_object() method removes a specific object from the collection using its unique id.

from endee import Endee

client = Endee(token="your-token-here")
collection = client.get_collection(name="my_collection")

# Delete a single object by ID
collection.delete_object("obj1")

Filtered Deletion

Delete objects based on filter fields when you don't know the exact object id.

from endee import Endee

client = Endee(token="your-token-here")
collection = client.get_collection(name="my_collection")

# Delete all objects matching filter conditions
collection.delete_with_filter([{"tags": {"$eq": "important"}}])

Update Filters

The collection.update_filters() method allows you to update the filters for specific objects without modifying the vector data or other metadata fields.

from endee import Endee

client = Endee(token="your-token-here")
collection = client.get_collection(name="my_collection")

# Update filters for multiple objects
collection.update_filters([
    {"id": "obj1", "filter": {"category": "B", "tags": "updated"}},
    {"id": "obj2", "filter": {"category": "C", "priority": 1}},
    {"id": "obj3", "filter": {"visibility": "private"}}
])

Parameters:

• updates: List of dictionaries, each containing:
  • id (str): Unique object identifier (required)
  • filter (dict): New filters to set (required)

Returns: Success message with the number of filters updated

Note: This operation only updates the filters. The vector data, metadata (meta), and other fields remain unchanged.

Describe Collection

# Get collection statistics and configuration info
info = collection.describe()

Hybrid Search

Hybrid search combines dense vector embeddings (semantic similarity) with sparse vectors (keyword/term matching) to provide more powerful and flexible search capabilities. This is particularly useful for applications that need both semantic understanding and exact term matching, such as:

• RAG (Retrieval-Augmented Generation) systems
• Document search with keyword boosting
• Multi-modal search combining different ranking signals
• BM25 + neural embedding fusion

Creating a Hybrid Collection

To enable hybrid search, specify the sparse_model parameter when creating a collection. This defines the model for the sparse vector space.

from endee import Endee, Precision

client = Endee(token="your-token-here")

client.create_collection(
    name="hybridtest1",
    dimension=384,              # dense vector dimension
    sparse_model="default",           # sparse vector model (default)
    space_type="cosine",
    precision=Precision.INT8   # Use Precision enum
)

# Get reference to the hybrid collection
collection = client.get_collection(name="hybridtest1")

Upserting Hybrid Objects

When upserting objects to a hybrid collection, you must provide both dense vectors and sparse vector representations. Sparse vectors are defined using two parallel arrays: sparse_indices (positions) and sparse_values (weights).

import numpy as np
import random

np.random.seed(42)
random.seed(42)

TOTAL_VECTORS = 2000
BATCH_SIZE = 1000
DIM = 384
SPARSE_DIM = 30000

batch = []

for i in range(TOTAL_VECTORS):
    # Dense vector (semantic embedding)
    dense_vec = np.random.rand(DIM).astype(float).tolist()
    
    # Sparse vector (e.g., BM25 term weights)
    # Example: 20 non-zero terms
    nnz = 20
    sparse_indices = random.sample(range(SPARSE_DIM), nnz)
    sparse_values = np.random.rand(nnz).astype(float).tolist()
    
    item = {
        "id": f"hybrid_vec_{i+1}",
        "vector": dense_vec,
        
        # Required for hybrid search
        "sparse_indices": sparse_indices,
        "sparse_values": sparse_values,
        
        "meta": {
            "title": f"Hybrid Document {i+1}",
            "index": i,
        },
        "filter": {
            "visibility": "public" if i % 2 == 0 else "private"
        }
    }
    
    batch.append(item)
    
    if len(batch) == BATCH_SIZE or i + 1 == TOTAL_VECTORS:
        collection.upsert(batch)
        print(f"Upserted {len(batch)} hybrid objects")
        batch = []

Hybrid Object Fields:

• id: Unique identifier (required)
• vector: Dense embedding vector (required)
• sparse_indices: List of non-zero term positions in sparse vector (required for hybrid)
• sparse_values: List of weights corresponding to sparse_indices (required for hybrid)
• meta: Metadata dictionary (optional)
• filter: Filter fields for structured filtering (optional)

Note: The lengths of sparse_indices and sparse_values must match.

Querying with Hybrid Search

Hybrid queries combine dense and sparse vector similarity to rank results. Provide both a dense query vector and sparse query representation.

import numpy as np
import random

np.random.seed(123)
random.seed(123)

DIM = 384
SPARSE_DIM = 30000

# Dense query vector (semantic)
dense_query = np.random.rand(DIM).astype(float).tolist()

# Sparse query
nnz = 15
sparse_indices = random.sample(range(SPARSE_DIM), nnz)
sparse_values = np.random.rand(nnz).astype(float).tolist()

results = collection.query(
    vector=dense_query,              # dense part
    sparse_indices=sparse_indices,   # sparse part
    sparse_values=sparse_values,
    top_k=5,
    ef=128,
    include_vectors=True
)

# Process results
for result in results:
    print(f"ID: {result['id']}")
    print(f"Similarity: {result['similarity']}")
    print(f"Metadata: {result['meta']}")
    print("---")

Hybrid Query Parameters:

• vector: Dense query vector (required)
• sparse_indices: Non-zero term positions in sparse query (required for hybrid)
• sparse_values: Weights for sparse query terms (required for hybrid)
• top_k: Number of results to return
• ef: Search quality parameter
• include_vectors: Include vector data in results (default: False)
• filter: Optional filter criteria
• dense_rrf_weight: Weight for dense ranking in RRF fusion (default: 0.5). See How Hybrid Ranking Works.
• rrf_rank_constant: RRF smoothing constant (default: 60). See How Hybrid Ranking Works.
• prefilter_cardinality_threshold: Controls when search switches from HNSW filtered search to brute-force prefiltering on the matched subset . See Filter Tuning for details.
• filter_boost_percentage: Expands the internal HNSW candidate pool by this percentage when a filter is active, compensating for filtered-out results . See Filter Tuning for details.

Hybrid Result Fields:

• id: Vector identifier
• similarity: Similarity score
• distance: Distance score (1.0 - similarity)
• meta: Metadata dictionary
• norm: Vector norm
• filter: Filter dictionary (if filter dict was included in upsert object while upserting)
• vector: Vector data (dense only) (if include_vectors=True)

How Hybrid Ranking Works: Reciprocal Rank Fusion (RRF)

When you run a hybrid query, the server independently ranks documents by dense similarity and sparse (BM25) score, then merges both lists using Reciprocal Rank Fusion (RRF):

rrf_score = dense_rrf_weight / (rrf_rank_constant + dense_rank)
          + (1 - dense_rrf_weight) / (rrf_rank_constant + sparse_rank)

Parameter	What it does	Default
dense_rrf_weight	Weight for dense ranking (0.0 = full sparse, 1.0 = full dense)	0.5
rrf_rank_constant	Smoothing constant — higher values flatten the score curve	60

results = collection.query(
    vector=dense_query,
    sparse_indices=sparse_indices,
    sparse_values=sparse_values,
    top_k=10,
    dense_rrf_weight=0.7,   # favor semantic results
    rrf_rank_constant=60,
)

Hybrid Search Use Cases

1. BM25 + Neural Embeddings

# Combine traditional keyword search (BM25) with semantic embeddings
# sparse_indices: term IDs from BM25
# sparse_values: BM25 scores
# vector: neural embedding from model like BERT

2. SPLADE + Dense Retrieval

# Use learned sparse representations (SPLADE) with dense embeddings
# sparse_indices/values: SPLADE model output
# vector: dense embedding from same or different model

3. Multi-Signal Ranking

# Combine multiple ranking signals
# sparse: user behavior signals, click-through rates
# dense: content similarity embedding

Filtered Querying

The collection.query() method supports structured filtering using the filter parameter. This allows you to restrict search results based on metadata conditions, in addition to vector similarity.

To apply multiple filter conditions, pass an array of filter objects, where each object defines a separate condition. All filters are combined with logical AND — meaning an object must match all specified conditions to be included in the results.

collection = client.get_collection(name="my_collection")

# Query with multiple filter conditions (AND logic)
filtered_results = collection.query(
    vector=[...],
    top_k=5,
    ef=128,
    include_vectors=True,
    filter=[
        {"tags": {"$eq": "important"}},
        {"visibility": {"$eq": "public"}}
    ]
)

Filtering Operators

The filter parameter supports a range of comparison operators to build structured queries.

Operator	Description	Supported Type	Example Usage
$eq	Matches values that are equal	String, Number	{"status": {"$eq": "published"}}
$in	Matches any value in the provided list	String	{"tags": {"$in": ["ai", "ml"]}}
$range	Matches values between a start and end value, inclusive	Number	{"score": {"$range": [70, 95]}}

Important Notes:

• Operators are case-sensitive and must be prefixed with a $
• Filters operate on fields provided under the filter key during object upsert

Filter Examples

# Equal operator - exact match
filter=[{"status": {"$eq": "published"}}]

# In operator - match any value in list
filter=[{"tags": {"$in": ["ai", "ml", "data-science"]}}]

# Range operator - numeric range (inclusive)
filter=[{"score": {"$range": [70, 95]}}]

# Combined filters (AND logic)
filter=[
    {"status": {"$eq": "published"}},
    {"tags": {"$in": ["ai", "ml"]}},
    {"score": {"$range": [80, 100]}}
]

Filter Tuning

When using filtered queries, two optional parameters let you tune the trade-off between search speed and recall:

prefilter_cardinality_threshold

Controls when the search strategy switches from HNSW filtered search (fast, graph-based) to brute-force prefiltering (exhaustive scan on the matched subset).

Value	Behavior
1_000	Prefilter only for very selective filters — minimum value
10_000	Prefilter only when the filter matches ≤10,000 vectors (default)
1_000_000	Prefilter for almost all filtered searches — maximum value

The intuition: when very few vectors match your filter, HNSW may struggle to find enough valid candidates through graph traversal. In that case, scanning the filtered subset directly (prefiltering) is faster and more accurate. Raising the threshold means prefiltering kicks in more often; lowering it favors HNSW graph search.

# Only prefilter when filter matches ≤5,000 vectors
results = collection.query(
    vector=[...],
    top_k=10,
    filter=[{"category": {"$eq": "rare"}}],
    prefilter_cardinality_threshold=5_000,
)

filter_boost_percentage

When using HNSW filtered search, some candidates explored during graph traversal are discarded by the filter, which can leave you with fewer results than top_k. filter_boost_percentage compensates by expanding the internal candidate pool before filtering is applied.

• 0 → no boost, standard candidate pool size (default)
• 20 → fetch 20% more candidates internally before applying the filter
• Maximum: 400 (doubles the candidate pool)

# Fetch 30% more candidates to compensate for aggressive filtering
results = collection.query(
    vector=[...],
    top_k=10,
    filter=[{"visibility": {"$eq": "public"}}],
    filter_boost_percentage=30,
)

Using Both Together

results = collection.query(
    vector=[...],
    top_k=10,
    filter=[{"category": {"$eq": "rare"}}],
    prefilter_cardinality_threshold=5_000,  # switch to brute-force for small match sets
    filter_boost_percentage=25,             # boost candidates for HNSW filtered search
)

Tip: Start with the defaults (prefilter_cardinality_threshold=10_000, filter_boost_percentage=0). If filtered queries return fewer results than expected, try increasing filter_boost_percentage. If filtered queries are slow on selective filters, try lowering prefilter_cardinality_threshold. Valid range for the threshold is 1,000–1,000,000.

---

API Reference

Endee Class

Method	Description
__init__(token=None)	Initialize client with optional API token
set_token(token)	Set or update API token
set_base_url(base_url)	Set custom API endpoint
create_collection(name, dimension, space_type, M, ef_con, precision, sparse_model)	Create a new vector collection
list_collections()	List all collections in workspace
delete_collection(name)	Delete a collection
get_collection(name)	Get reference to a collection

Object Methods

Method	Description
upsert(input_array)	Insert or update objects (max 10000 per batch)
query(vector, top_k, filter, ef, include_vectors, sparse_indices, sparse_values, dense_rrf_weight, rrf_rank_constant, prefilter_cardinality_threshold, filter_boost_percentage)	Search for similar objects
get_object(id)	Get a specific object by ID
delete_object(id)	Delete an object by ID
delete_with_filter(filter)	Delete objects matching a filter
update_filters(updates)	Update filters for multiple objects by ID
describe()	Get collection statistics and configuration

Precision Enum

The Precision enum provides type-safe precision levels for vector quantization:

from endee import Precision

# Available values
Precision.BINARY2   # 1-bit binary quantization
Precision.INT8     # 8-bit integer quantization 
Precision.INT16    # 16-bit integer quantization (default)
Precision.FLOAT16   # 16-bit floating point
Precision.FLOAT32   # 32-bit floating point

License

MIT License