DXMT Python SDK

Official Python SDK for DXMT VectorDB - a lightweight single-node vector database written in Rust.

Features

• 🚀 Simple API - Clean, intuitive Python interface
• ⚡ Async Support - Full async/await support for high performance
• 🔍 Vector Search - Multiple distance metrics (Cosine, Euclidean, Manhattan, Dot Product)
• 📊 Multiple Index Types - HNSW, IVF, IVF-PQ, LSH, Brute Force
• 📝 Full-Text Search - BM25 with Tantivy integration
• 🔀 Hybrid Search - Combine vector similarity + full-text search
• 🇨🇳 Chinese Support - Jieba tokenizer for Chinese text
• 🎯 Type Safe - Full type hints with mypy support
• 📦 Zero Dependencies - Only requires httpx

⚠️ Important Note (Updated 2026-01-22): When using full-text search or hybrid search, you must use the content field in your payload for main text content. Using other field names like "text" will cause full-text search to return 0 results. See Field Naming Conventions for details.

Installation

pip install dxmt-python-sdk

The package is not on PyPI yet. Until the packages are published, install from source:

pip install "git+https://github.com/dxmt/dxmt#subdirectory=vectordb/clients/dxmt-python-sdk"
# or, from a checkout of the repository:
pip install ./vectordb/clients/dxmt-python-sdk

Quick Start

Basic Usage

from dxmt import Client

# Connect to DXMT server
client = Client(host="localhost", port=9999)

# Create a table
client.create_table(
    name="documents",
    dimension=1536,
    distance_metric="cosine",
    index_type="hnsw"
)

# Insert vectors
client.insert_records(
    table_name="documents",
    records=[
        {
            "id": "doc1",
            "vector": [0.1, 0.2, 0.3, ...],  # 1536-dim vector
            "payload": {
                "title": "Document 1",
                "content": "This is document 1",
                "source": "file.txt"
            }
        },
        {
            "id": "doc2",
            "vector": [0.4, 0.5, 0.6, ...],
            "payload": {
                "title": "Document 2",
                "content": "This is document 2",
                "source": "file.txt"
            }
        }
    ]
)

# Search for similar vectors
results = client.search(
    table_name="documents",
    vector=[0.1, 0.2, 0.3, ...],  # query vector
    limit=10
)

for result in results:
    print(f"ID: {result['id']}, Score: {result['score']}")

Authentication

A server started with DXMT_API_KEY answers 401 to everything but its health probes until the key is sent. It goes out as the api-key header; send it over TLS, or it is readable at every hop.

client = Client(host="db.example.com", ssl=True, api_key="your-key")

The client reads no environment variable itself — pass api_key=os.environ["DXMT_API_KEY"] if that is where you keep it. Over plain HTTP to anything but localhost it warns.

Loading and Exporting a Table

upload inserts any iterable of records — a generator included, drawn a batch at a time — in batches, in parallel if asked, retrying what a retry can fix (a dropped connection, a timeout, 429, a 5xx) and reporting what could not go in rather than raising halfway:

result = client.upload("docs", records, batch_size=256, parallel=4)
if not result.ok:
    for failure in result.failed:
        print(failure.status, failure.error, failure.ids)

scroll reads every record back, newest first, a page at a time. It pages by cursor, so every record present for the whole scroll comes back exactly once even while the table is written to — where list_records with an offset shifts under inserts and repeats or skips records:

for record in client.scroll("docs", batch_size=500, filter={"must": [...]}):
    ...

Async Usage

import asyncio
from dxmt import Client

async def main():
    client = Client(host="localhost", port=9999)

    # Create table
    await client.acreate_table(
        name="documents",
        dimension=1536,
        distance_metric="cosine"
    )

    # Insert records
    await client.ainsert_records(
        table_name="documents",
        records=[...]
    )

    # Search
    results = await client.asearch(
        table_name="documents",
        vector=[0.1, 0.2, ...],
        limit=10
    )

    await client.aclose()

asyncio.run(main())

Context Manager

from dxmt import Client

# Automatic cleanup
with Client(host="localhost", port=9999) as client:
    client.create_table(name="test", dimension=128)
    # ... do work ...

# Async context manager
async with Client(host="localhost", port=9999) as client:
    await client.acreate_table(name="test", dimension=128)
    # ... do work ...

Hybrid Search (Vector + Text)

from dxmt import Client

client = Client(host="localhost", port=9999)

# Create table with full-text search enabled
client.create_table(
    name="documents",
    dimension=1536,
    distance_metric="cosine",
    enable_full_text_search=True  # Enable text search
)

# Insert documents with "content" field for text search
client.insert_records(
    table_name="documents",
    records=[
        {
            "id": "doc1",
            "vector": embedding1,
            "payload": {
                "title": "Machine Learning Guide",
                "content": "Machine learning is...",  # Required for text search
                "category": "AI"
            }
        }
    ]
)

# Hybrid search: combine vector similarity + text matching
results = client.hybrid_search(
    table_name="documents",
    vector=[0.1, 0.2, ...],          # Vector query
    text_query="machine learning",   # Text query
    vector_weight=0.5,                # 0=text only, 1=vector only; omit to let the server fuse with dbsf
    limit=10
)

Table Management

from dxmt import Client

client = Client()

# List all tables
tables = client.list_tables()
print(f"Tables: {tables}")

# Get table info
info = client.get_table("documents")
print(f"Dimension: {info['dimension']}")
print(f"Distance: {info['distance_metric']}")

# Delete table
client.delete_table("documents")

Record Operations

# Get records by IDs
records = client.get_records(
    table_name="documents",
    ids=["doc1", "doc2"]
)

# Delete records
client.delete_records(
    table_name="documents",
    ids=["doc1", "doc2"]
)

Field Naming Conventions

Standard Payload Fields

DXMT uses specific field names for optimal functionality, especially when using full-text search and hybrid search:

Field Name	Purpose	Required	Example
content	Main searchable text content	Yes for text search	Article body, document content
title	Document title (higher search boost)	No	"Introduction to AI"
description	Brief description/summary	No	"An overview of..."
tags	Categorization tags	No	["AI", "ML"]

⚠️ Important: Use content for Text Search

When using full-text search or hybrid search, you must use the content field for your main text:

# ✅ CORRECT - Use "content"
records = [{
    "id": "doc1",
    "vector": [0.1, 0.2, ...],
    "payload": {
        "title": "Introduction to Machine Learning",
        "content": "Machine learning is a subset of AI...",  # ✅
        "category": "AI"
    }
}]

# ❌ WRONG - Don't use "text" or other field names
records = [{
    "id": "doc1",
    "vector": [0.1, 0.2, ...],
    "payload": {
        "title": "Introduction to Machine Learning",
        "text": "Machine learning is...",  # ❌ Won't work!
        "category": "AI"
    }
}]

Why Field Names Matter

The full-text search engine (Tantivy) maps payload fields to its internal schema:

• content → Main searchable field (BM25 ranking)
• title → Title field (2x boost in search)
• description → Description field (0.8x boost)
• tags → Tags field (1.5x boost, supports filtering)

If you use incorrect field names:

• Full-text search will return 0 results
• Hybrid search text component will not work
• No error is thrown (silent failure)

Full-Text Search Example

# Create table with full-text search
client.create_table(
    name="articles",
    dimension=128,
    distance_metric="cosine",
    enable_full_text_search=True  # Enable text search
)

# Insert with correct field names
client.insert_records(
    table_name="articles",
    records=[
        {
            "id": "art1",
            "vector": embedding1,
            "payload": {
                "title": "Deep Learning Fundamentals",
                "content": "Deep learning uses neural networks...",  # ✅ Required
                "category": "AI",
                "tags": ["deep-learning", "neural-networks"]
            }
        }
    ]
)

# Full-text search
results = client.full_text_search(
    table_name="articles",
    text_query="neural networks",
    limit=10
)

# Hybrid search (vector + text)
results = client.hybrid_search(
    table_name="articles",
    vector=query_embedding,
    text_query="deep learning",
    vector_weight=0.5,  # 0.0 = text only, 1.0 = vector only; omit for the server's dbsf
    limit=10
)

Custom Metadata Fields

You can add any custom fields alongside the standard fields:

"payload": {
    "content": "Main text here...",      # ✅ Required for text search
    "title": "Document Title",           # Standard field
    "author": "John Doe",                # ✅ Custom field
    "publish_date": "2024-01-15",       # ✅ Custom field
    "rating": 4.5,                       # ✅ Custom field
    "custom_metadata": {...}             # ✅ Any structure
}

Migration from Old Code

If your code uses "text" field, update it to "content":

# Old code (before 2026-01-22)
"payload": {"text": "..."}

# New code (after 2026-01-22)
"payload": {"content": "..."}  # ✅ Required change

After updating field names, you need to:

1. Re-create tables with full-text search
2. Re-insert all records with corrected field names

Advanced Features

Distance Metrics

# Choose your distance metric
client.create_table(
    name="table",
    dimension=128,
    distance_metric="cosine"      # or "euclidean", "manhattan", "dot"
)

Index Types

# Choose optimal index for your use case
client.create_table(
    name="table",
    dimension=128,
    index_type="hnsw"  # Fast, high recall
    # or: "ivf", "ivf_pq" (memory efficient), "lsh", "brute_force"
)

Insert Modes

# One call is one request, so bigger batches amortise the round trip
client.insert_records(table_name="documents", records=[...])

# Or let upload() batch, parallelise and retry for you
client.upload("documents", records, batch_size=256, parallel=4)

Filtering

# Filter search results by metadata
results = client.search(
    table_name="documents",
    vector=[0.1, 0.2, ...],
    limit=10,
    filter={
        "source": {"$eq": "file.txt"},
        "date_ts": {"$gte": 1704067200}  # ranges compare numbers: 2024-01-01 as a timestamp
    }
)

Integrations

Coming from Chroma

python -m dxmt.migrate.chroma copies Chroma collections into DXMT tables with their stored vectors, so nothing is re-embedded: the same ids, the collection's distance (l2 becomes euclidean, ip becomes dot), each metadata dict as the payload, and the document under text with a full-text index on it. Records are written by id, so a second run over the same table rewrites them rather than duplicating them, and an interrupted copy is finished by running it again.

pip install 'dxmt-python-sdk[chroma]'   # until published: pip install './vectordb/clients/dxmt-python-sdk[chroma]'
python -m dxmt.migrate.chroma --path ./chroma_db                       # every collection
python -m dxmt.migrate.chroma --path ./chroma_db --collection notes --table notes --analyzer chinese
python -m dxmt.migrate.chroma --chroma-host localhost:8000 --url http://localhost:9999

mem0 Integration

DXMT can be used as a vector storage backend for mem0, an intelligent memory layer for AI agents.

Installation:

pip install mem0ai dxmt-python-sdk   # until published: pip install mem0ai ./vectordb/clients/dxmt-python-sdk

Usage:

from mem0 import Memory
from dxmt import Client
from dxmt.mem0 import DXMTMemoryStore

# Initialize DXMT client
client = Client(host="localhost", port=9999)

# Create DXMT memory store
vector_store = DXMTMemoryStore(
    collection_name="user_memories",
    client=client,
    embedding_model_dims=1536  # OpenAI text-embedding-ada-002
)

# Configure mem0
config = {
    "vector_store": {
        "provider": "custom",
        "config": {"custom_vector_store": vector_store}
    },
    "embedder": {
        "provider": "openai",
        "config": {"model": "text-embedding-ada-002"}
    }
}

m = Memory.from_config(config)

# Store memories
m.add("I love Python programming", user_id="user_123")

# Search memories
results = m.search("programming preferences", user_id="user_123")
for mem in results:
    print(f"{mem['memory']} (score: {mem['score']:.3f})")

Features:

• Multi-scope memory (user_id, agent_id, run_id)
• Automatic deduplication via hash
• Efficient filtering and semantic search
• Timestamp tracking (created_at, updated_at)

API Reference

Client

__init__(host="localhost", port=9999, ssl=False, timeout=30.0)

Create a new DXMT client.

Parameters:

• host (str): Server hostname
• port (int): Server port
• ssl (bool): Use HTTPS
• timeout (float): Request timeout in seconds

create_table(name, dimension, distance_metric="cosine", index_type="hnsw", **kwargs)

Create a new table.

Parameters:

• name (str): Table name
• dimension (int): Vector dimension
• distance_metric (str): Distance metric ("cosine", "euclidean", "manhattan", "dot")
• index_type (str): Index type ("hnsw", "ivf", "ivf_pq", "lsh", "brute_force")

insert_records(table_name, records)

Insert records into a table.

Parameters:

• table_name (str): Table name
• records (list[dict]): Records to insert (each record must have id, vector, and payload)

Record Structure:

{
    "id": "unique_id",           # Required: Record ID
    "vector": [0.1, 0.2, ...],   # Required: Embedding vector
    "payload": {                 # Required: Metadata
        "content": "...",        # For full-text search
        "title": "...",          # Optional
        # ... custom fields
    }
}

search(table_name, vector=None, text=None, k=10, filter=None, search_mode=None, offset=0, mmr=None, ...)

Search for similar vectors. limit is accepted as an alias for k.

Parameters:

• table_name (str): Table name
• vector (list[float]): Query vector -- or text (str) on a table with a server-side embedding model
• k (int): Number of results (default 10)
• filter (dict): Metadata filter
• search_mode (str): fast / normal / accurate / exact; exact scores every row and is the ground truth the others approximate
• mmr (dict): {"lambda_mult": 0.5, "fetch_k": 50} re-ranks the candidates for diversity

hybrid_search(table_name, vector=None, text=None, vector_weight=None, fusion_method=None, k=10, filter=None)

Hybrid search combining vector + text. Needs a server built with text-bm25; a default build refuses a request carrying text with FEATURE_DISABLED rather than answering from the vector alone. text_query, alpha and limit are accepted as aliases for text, vector_weight and k.

Parameters:

• table_name (str): Table name
• vector (list[float]): Query vector
• text (str): Text query
• vector_weight (float): 0 = text only, 1 = vector only. No default: when omitted, nothing is sent and the server fuses with dbsf, which normalises each side by its own score distribution and has no weight to tune. Passing a weight selects weighted_sum unless fusion_method names another method (reciprocal_rank_fusion also reads the weight).
• k (int): Number of results

Async Methods

All methods have async equivalents with a prefix:

• acreate_table(), ainsert_records(), asearch(), etc.

Metadata Filters

DXMT supports flexible metadata filtering with MongoDB-style operators. See FILTERS_GUIDE.md for comprehensive documentation.

Quick examples:

# Simple equality
filter = {"category": "AI", "year": 2023}

# Comparison operators
filter = {"rating": {"$gte": 8.0}, "year": {"$lt": 2024}}

# Logical operators
filter = {
    "$and": [
        {"year": {"$gte": 2022}},
        {"rating": {"$gte": 8.0}},
        {"category": {"$in": ["AI", "ML", "DL"]}}
    ]
}

# Use in search
results = client.search(
    table_name="documents",
    vector=query_vector,
    limit=10,
    filter=filter
)

# Or build the same thing with Field
from dxmt import Field

filter = (Field("year") >= 2022) & (Field("rating") >= 8.0) & Field("category").is_in(["AI", "ML"])

A filter that has no exact translation -- (a and b) or c, two $or groups, a range with a string bound -- raises ValueError rather than running as a different query.

Supported operators:

• Comparison: $eq, $ne, $gt, $gte, $lt, $lte
• Array: $in, $nin
• String: $contains, $startswith, $endswith
• Logical: $and, $or, $nor ($not with a list is the same)

Development

Setup

# Clone repository
git clone https://github.com/dxmt/dxmt.git
cd dxmt/vectordb/clients/dxmt-python-sdk

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in development mode
pip install -e ".[dev]"

Testing

# Run tests
pytest

# Run with coverage
pytest --cov=dxmt --cov-report=term-missing

# Run specific test file
pytest tests/test_client.py -v

Code Quality

# Lint
ruff check dxmt

# Format
ruff format dxmt

# Type check
mypy dxmt

Requirements

• Python 3.9+
• DXMT VectorDB server (running locally or remotely)

License

Apache License 2.0 - see LICENSE file for details.

Links

• Homepage: https://github.com/dxmt/dxmt
• Documentation: https://docs.dxmt.online
• Issues: https://github.com/dxmt/dxmt/issues
• PyPI: https://pypi.org/project/dxmt-python-sdk/

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.