endee 0.1.37b1

Endee is the Next-Generation Vector Database for Scalable, High-Performance AI

Endee — Python Client

Endee is a high-performance C++ vector database for AI search, RAG, and hybrid retrieval. A collection holds objects, and each object can carry values for several named, typed fields at once — dense vectors, sparse vectors, and multi-vectors — so a single object can be searched many ways.

• Multi-field objects — one object, many vector fields (dense + sparse + multi-vector).
• Hybrid search — query any subset of fields; get per-field results, or fuse them with optional RRF.
• Client-side normalization — cosine vectors are L2-normalized for you.
• Full control plane — collections, objects, filters, rebuild/shrink, backups, databases, and tokens.

---

Table of contents

1. Install
2. Quick start
3. Connecting & tokens
4. Create a collection
5. Insert objects (combining vector types)
6. Search
   • Single field
   • Per-field options
   • Multi-field without a reranker
   • Multi-field with RRF
   • Filters
7. Object operations
8. Collection maintenance
9. Backups
10. Database administration (root token)
11. Self-service tokens
12. Server info
13. Reference
14. Error handling

---

Install

pip install endee

Requires Python ≥ 3.9. Depends on numpy, msgpack, orjson, requests, httpx.

---

Quick start

from endee import Endee

# A "db token" (db_name:secret) scopes you to one database.
client = Endee(token="alice:xxxxxxxx")
client.set_base_url("http://localhost:8080/api/v2")   # include /api/v2

# 1. Create a collection with one of each field type.
client.create_collection(
    name="products",
    fields=[
        {"name": "embedding", "type": "vector",
         "params": {"dimension": 8, "space_type": "cosine", "precision": "int8"}},
        {"name": "keywords", "type": "sparse", "sparse_model": "default"},
        {"name": "colbert", "type": "multi_vector",
         "params": {"dimension": 8, "space_type": "cosine", "precision": "int8",
                    "pooling": "mean"}},
    ],
)
collection = client.get_collection("products")

# 2. Insert an object carrying ALL THREE field types at once.
collection.upsert([{
    "id": "p1",
    "meta": {"name": "Wireless Headphones", "price": 99},
    "filter": {"category": "electronics"},
    "fields": {
        "embedding": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],
        "keywords":  {"indices": [3, 17, 42], "values": [0.9, 0.5, 0.2]},
        "colbert":   [[0.1]*8, [0.2]*8],
    },
}])

# 3. Search the dense field.
hits = collection.search(fields={"embedding": [0.2]*8}, limit=5)
for h in hits["results"]:
    print(h["id"], h["similarity"], h["meta"])

---

Connecting & tokens

Authentication is always required. The server has no anonymous mode — every request must carry a token, or it is rejected with 401 Unauthorized. (Only health() and stats() are unauthenticated.)

Default root token (dev): if the server is started without NDD_ROOT_TOKEN, it falls back to the well-known insecure token unsafe_root_token (and logs a loud startup warning). Use that value as your root token for all root/admin operations on such a server:

admin = Endee(token="unsafe_root_token")
admin.set_base_url("http://localhost:8080/api/v2")

This is for local/dev only — production must set NDD_ROOT_TOKEN.

There are two kinds of token:

Token	Format	Used for
root token	the server's NDD_ROOT_TOKEN	Database administration (create/delete databases, mint tokens). Always required to bootstrap — see below.
db token	db_name:secret	All collection / object / search / backup work inside one database.

You always start from the root token

There is no default database. Before any data work exists, you must use the root token to create a database — which returns its first db token. So the root token is the entry point to every deployment:

# 1) Admin client — uses the ROOT token (the server's NDD_ROOT_TOKEN).
admin = Endee(token="roottoken")
admin.set_base_url("http://localhost:8080/api/v2")

# 2) Create a database; this returns the db token apps will use.
db_token = admin.create_database("alice", db_type="scale")["db_token"]

# 3) App client — uses that DB token for all collection/data work.
client = Endee(token=db_token)
client.set_base_url("http://localhost:8080/api/v2")

Notes:

• The root token is the sole admin. It cannot run data ops directly — it administers databases/tokens, and a db token (which it mints) does the data work.
• set_base_url must point at the /api/v2 root of your server.
• A missing/invalid token → AuthenticationException (401); a read-only token on a write → ForbiddenException (403).

---

Create a collection

A collection is a set of named, typed fields. Each field is a plain dict sent to the server as-is.

Field type	Required keys	params
vector	name, type	dimension, space_type, precision (+ optional M, ef_con)
sparse	name, type, sparse_model	— (no params)
multi_vector	name, type	dimension, space_type, precision, pooling ("mean"/"max") (+ optional M, ef_con)

client.create_collection(
    name="products",
    fields=[
        # Dense embedding (single vector per object)
        {"name": "embedding", "type": "vector",
         "params": {"dimension": 768, "space_type": "cosine", "precision": "int8"}},

        # Sparse / keyword field (BM25-style)
        {"name": "keywords", "type": "sparse", "sparse_model": "default"},

        # Multi-vector field (many vectors per object)
        {"name": "colbert", "type": "multi_vector",
         "params": {"dimension": 768, "space_type": "cosine", "precision": "int8",
                    "pooling": "mean"}},
    ],
)

• space_type: cosine (default), l2, ip.
• precision: float32, float16, int16 (default), int8, int8e, binary.
• M / ef_con: HNSW build params (optional; sensible defaults applied).
• Cosine vectors are L2-normalized by the client before sending.

Other client-level collection calls:

client.list_collections()              # ["products", ...]
collection = client.get_collection("products")
collection.describe()                  # {name, fields, created_at, layout_version}
client.delete_collection("products")

---

Insert objects (combining vector types)

collection.upsert(objects) takes a list of objects. Each object is:

{
    "id":     "<unique string id>",
    "meta":   {...},          # arbitrary JSON, returned in search results (optional)
    "filter": {...},          # tag dict used by filtered search (optional)
    "fields": {               # field_name -> value, ONE entry per field you set
        "<dense_field>":  [float, ...],                       # dense   → list
        "<sparse_field>": {"indices": [...], "values": [...]},# sparse  → dict
        "<multi_field>":  [[float, ...], [float, ...], ...],  # multi   → list of lists
    },
}

Per-field value formats:

Field type	Value format
vector	[0.1, 0.2, ...] — a flat list of floats
sparse	{"indices": [3, 17], "values": [0.9, 0.5]} (also accepts sparse_indices/sparse_values)
multi_vector	[[...], [...], ...] — a list of equal-length float lists

One object, multiple field types

A single object can populate any combination of the collection's fields in one call. They're stored together under the same id and can be searched independently or together:

collection.upsert([
    {
        "id": "p1",
        "meta": {"name": "Wireless Headphones", "price": 99},
        "filter": {"category": "electronics"},
        "fields": {
            "embedding": [0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80],   # dense
            "keywords":  {"indices": [3, 17, 42], "values": [0.9, 0.5, 0.2]}, # sparse
            "colbert":   [                                                     # multi-vector
                [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],
                [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
            ],
        },
    },
    {
        "id": "p2",
        "meta": {"name": "Running Shoes"},
        "filter": {"category": "footwear"},
        "fields": {
            # An object may set only SOME fields — here just dense + sparse.
            "embedding": [0.50, 0.40, 0.30, 0.20, 0.10, 0.05, 0.02, 0.01],
            "keywords":  {"indices": [5, 17, 90], "values": [0.7, 0.6, 0.1]},
        },
    },
])
# → {"upserted": 2}

Notes:

• You don't have to fill every field on every object — set whatever subset you have.
• upsert is insert-or-replace by id.
• Cosine vector/multi_vector values are normalized client-side before sending.
• Validation is done client-side (fails fast): batch size ≤ 10,000, no duplicate ids in a batch, dense/multi dims must match the field, sparse indices/values lengths must match, filter keys ≤ 128 bytes / string values ≤ 1024 bytes.

---

Search

collection.search(fields=..., limit=..., reranker=...) queries one or more fields in a single request. You query a field with the same shape you'd insert:

Field type	Query shape
vector	[0.2, 0.2, ...]
sparse	{"indices": [3, 17], "values": [0.8, 0.4]}
multi_vector	[[...], [...]]

Each result hit is:

{"id": "p1", "similarity": 0.94, "meta": {...}, "filter": {...}}

Search results carry meta/filter but not the stored vectors. To get the vectors back, use get_objects.

Single-field search

When you query exactly one field, results is a ranked list:

res = collection.search(fields={"embedding": [0.2]*8}, limit=3)
res["results"]
# [
#   {"id": "p1", "similarity": 0.97, "meta": {...}, "filter": {...}},
#   {"id": "p2", "similarity": 0.88, "meta": {...}, "filter": {...}},
# ]

# sparse
collection.search(fields={"keywords": {"indices": [3, 17], "values": [0.8, 0.4]}}, limit=3)

# multi-vector
collection.search(fields={"colbert": [[0.1]*8, [0.2]*8]}, limit=3)

Per-field options (limit / ef_search)

Pass a config dict per field to control how many candidates that field fetches and its HNSW ef_search:

collection.search(fields={
    "embedding": {"query": [0.2]*8, "limit": 50, "ef_search": 256},
    "keywords":  {"query": {"indices": [3, 17], "values": [0.8, 0.4]}, "limit": 20},
})

Multi-field search without a reranker

When you query multiple fields and don't pass a reranker, you get each field's own ranked list, unfused — results is a dict keyed by field name. This lets you merge or display them however you like.

res = collection.search(
    fields={
        "embedding": {"query": [0.2, 0.2, 0.3, 0.3], "limit": 2},
        "keywords":  {"query": {"indices": [3, 17], "values": [0.8, 0.4]}, "limit": 2},
    },
    limit=2,
)

{
  "results": {
    "embedding": [
      {"id": "p1", "meta": {"name": "Wireless Headphones"},
       "filter": {"category": "electronics"}, "similarity": 0.4938},
      {"id": "p2", "meta": {"name": "Running Shoes"},
       "filter": {"category": "footwear"},    "similarity": 0.4505}
    ],
    "keywords": [
      {"id": "p1", "meta": {"name": "Wireless Headphones"},
       "filter": {"category": "electronics"}, "similarity": 0.9200},
      {"id": "p2", "meta": {"name": "Running Shoes"},
       "filter": {"category": "footwear"},    "similarity": 0.5591}
    ]
  }
}

res["results"]["embedding"]   # this field's ranked hits
res["results"]["keywords"]    # that field's ranked hits

similarity is in each field's own scale (cosine for dense ≈ 0.49, sparse dot for keywords ≈ 0.92) — scores are not comparable across fields. That's exactly why fusion is opt-in.

Multi-field search with RRF

Pass reranker="rrf" to fuse the per-field lists into one ranked list using Reciprocal Rank Fusion. Now results is a flat list with a single fused score:

res = collection.search(
    fields={
        "embedding": {"query": [0.2, 0.2, 0.3, 0.3], "limit": 50},
        "keywords":  {"query": {"indices": [3, 17], "values": [0.8, 0.4]}, "limit": 50},
        "colbert":   {"query": [[0.1]*8, [0.2]*8], "limit": 50},
    },
    limit=10,
    reranker="rrf",
    # optional: weight each field's contribution (must sum to 1.0; equal by default)
    field_weights={"embedding": 0.5, "keywords": 0.3, "colbert": 0.2},
    # optional: RRF rank constant k (default 60)
    rrf_k=60,
)

{
  "results": [
    {"id": "p1", "similarity": 0.0164, "meta": {...}, "filter": {...}},
    {"id": "p3", "similarity": 0.0160, "meta": {...}, "filter": {...}},
    {"id": "p2", "similarity": 0.0160, "meta": {...}, "filter": {...}}
  ]
}

Summary of return shapes

Query	reranker	results shape
1 field	(any)	[ hit, ... ]
N fields	None (default)	{ field_name: [ hit, ... ], ... }
N fields	"rrf"	[ hit, ... ] (fused)

reranker accepts only None or "rrf". field_weights / rrf_k are used only when reranker="rrf".

Filtered search

filter applies to any search mode. It's an array of conditions:

collection.search(
    fields={"embedding": [0.2]*8},
    limit=5,
    filter=[{"category": {"$eq": "electronics"}}],
)

Operators: $eq, $in, $range, $gt, $gte, $lt, $lte (see Reference).

---

Object operations

# Fetch full stored objects by id (meta, filter, and the stored vectors).
objs = collection.get_objects(["p1", "p2"])
# [
#   {"id": "p1", "meta": {...}, "filter": {...},
#    "vectors": {"embedding": [...]},               # dense fields (normalized)
#    "sparses": {"keywords": {"indices": [...], "values": [...]}},
#    "multi_vectors": {"colbert": [[...], [...]]}},
#   ...
# ]

# Delete a single object by id.
collection.delete_object("p1")                      # {"deleted": "p1"}

# Delete every object matching a filter.
collection.delete_by_filter([{"category": {"$eq": "footwear"}}])   # {"deleted": <count>}

# Update only the filter tags on existing objects (no re-upsert of vectors).
collection.update_filters([
    {"id": "p2", "filter": {"category": "sale"}},
])                                                  # {"updated": <count>}

---

Collection maintenance

collection.describe()                 # {name, fields, created_at, layout_version}

# Rebuild a field's HNSW graph with new M / ef_con (async).
collection.rebuild(field="embedding", m=24, ef_con=200)   # {status: "in_progress", ...}
collection.rebuild_status()                               # {percent_complete, ...}

# Defragment storage in place.
collection.shrink()                   # {"status": "ok", "reclaimed_bytes": <int>}

---

Backups

Backups are per-database. Creating one is per-collection and asynchronous.

import time

# Start a backup of a collection (async).
collection.create_backup("nightly")          # {"backup_name": "nightly", "status": "in_progress"}

# Poll for completion.
while client.active_backup().get("active"):
    time.sleep(0.5)

client.list_backups()                         # {backup_name: {metadata}, ...}
client.backup_info("nightly")                 # metadata for one backup

# Restore into a NEW collection.
client.restore_backup("nightly", "products_restored")

# Move backups around.
client.download_backup("nightly", "/tmp/nightly.tar")   # writes a .tar, returns the path
client.upload_backup("/tmp/nightly.tar")                # uploads a .tar into this db

client.delete_backup("nightly")

---

Database administration (root token)

These operate across databases and require the root token (the server's NDD_ROOT_TOKEN). Create an admin client with that token:

admin = Endee(token="roottoken")
admin.set_base_url("http://localhost:8080/api/v2")

Databases

# Create a database — returns a default db token you hand to an app.
res = admin.create_database("alice", db_type="scale")
db_token = res["db_token"]                     # e.g. "alice:8w12..."
# res == {"db_name": "alice", "db_type": "Scale", "db_token": "...",
#         "name": "default", "message": "Db created successfully"}

admin.list_databases()
# [{"db_name": "alice", "db_type": "Scale", "is_active": True, "created_at": 178...}, ...]

admin.get_database("alice")
# {"db_name": "alice", "db_type": "Scale", "is_active": True,
#  "created_at": 178..., "token_count": 1}

admin.set_database_type("alice", "pro")        # change tier
admin.deactivate_database("alice")             # block its tokens (data is kept)
admin.activate_database("alice")               # re-enable
admin.delete_database("alice")                 # delete db + ALL its data

db_type is one of starter, pro, scale, enterprise (each tier carries per-tier limits on objects, collections, dimension, etc.).

Collections across databases (admin view)

admin.list_db_collections("alice")             # ["products", ...] for one db
admin.list_all_collections()                   # [{"db_name": "alice", "collections": [...]}, ...]
admin.delete_db_collection("alice", "products")

Tokens for any database

# Create a named token for a db. token_type: "rw" (default) or "r" (read-only).
admin.create_token("alice", name="ingest", token_type="rw")
# {"db_token": "alice:...", "name": "ingest", "db_name": "alice", "token_type": "rw"}

admin.list_tokens("alice")
# [{"name": "ingest", "db_name": "alice", "token_type": "rw", "created_at": 178...}, ...]

admin.delete_token("alice", "ingest")

A read-only token (token_type="r") gets a 403 on any mutating request (upsert, delete, create collection, …).

Typical admin → app flow

# 1. Admin (root token) provisions a database and gets its token.
admin = Endee(token="roottoken"); admin.set_base_url(".../api/v2")
db_token = admin.create_database("alice", db_type="scale")["db_token"]

# 2. The app uses that db token for all data work.
app = Endee(token=db_token); app.set_base_url(".../api/v2")
app.create_collection("products", fields=[...])

---

Self-service tokens

A db token holder can manage their own database's tokens without the root token (e.g. an app team rotating its own keys):

client = Endee(token="alice:xxxx"); client.set_base_url(".../api/v2")

client.create_my_token("worker", token_type="r")   # {"db_token": "alice:...", "token_type": "r"}
client.list_my_tokens()                             # [{"name": "worker", "token_type": "r", ...}, ...]
client.delete_my_token("worker")

---

Server info

client.health()    # {"status": "ok", "timestamp": ...}
client.stats()     # {"version": "...", "uptime": ..., "total_requests": ...}

---

Reference

Distance / space types (space_type)

Value	Meaning
cosine	Cosine similarity (vectors L2-normalized client-side). Default.
l2	Euclidean distance.
ip	Inner product.

Precisions (precision) — memory/accuracy trade-off

float32, float16, int16 (default), int8, int8e, binary.

Database tiers (db_type): starter, pro, scale, enterprise.

Token types (token_type): rw (read-write), r (read-only).

Filter operators

Operator	Example	Meaning
$eq	{"category": {"$eq": "news"}}	equals
$in	{"tag": {"$in": ["a", "b"]}}	in set
$range	{"price": {"$range": [10, 100]}}	inclusive range
$gt / $gte	{"price": {"$gt": 50}}	greater than / or equal
$lt / $lte	{"price": {"$lte": 100}}	less than / or equal

Client-side validation / limits

Limit	Value
Objects per upsert	≤ 10,000
limit (top-k)	1 … 4096
ef_search	1 … 1024
Filter key	≤ 128 bytes
Filter string value	≤ 1024 bytes
Vector dimension	must match the field's configured dimension

---

Error handling

Non-2xx responses raise typed exceptions:

from endee import (
    EndeeException,           # base class for all client errors
    APIException,             # generic 4xx (e.g. bad request)
    AuthenticationException,  # 401
    ForbiddenException,       # 403 (e.g. read-only token on a write)
    NotFoundException,        # 404
    ConflictException,        # 409 (e.g. duplicate collection)
    ServerException,          # 5xx
)

try:
    collection.upsert(objects)
except NotFoundException as e:
    print("not found:", e)
except EndeeException as e:           # catches all of the above
    print("request failed:", e)

All exception classes are importable from endee (and endee.exceptions): EndeeException (base), APIException, AuthenticationException, ForbiddenException, NotFoundException, ConflictException, ServerException, SubscriptionException.

Client-side input mistakes (bad dimensions, mismatched sparse lengths, oversized batch, sum-of-weights ≠ 1.0, etc.) raise ValueError before any network call.

---

End-to-end examples

See scripts/:

• demo.py — full collection walkthrough (create → upsert → every search mode → backups → cleanup). Run with a db token: python3 scripts/demo.py --token alice:xxxx.
• demo_admin.py — database administration. Run with the root token: python3 scripts/demo_admin.py --root-token roottoken.
• test_bulk_100k.py — bulk ingest + self-recall benchmark.