Skip to main content

AkshamDB — Fast, Persistent, Embeddable Vector Database

v0.1.0 — A self-hosted vector database built from scratch in Python and C++. Designed for semantic search and RAG applications where you want to own the entire stack.


What Is This?

AkshamDB combines three production systems under one engine:

  • LSM Store — Write-Ahead Log (with fsync) → Memtable → SSTables with Bloom filters and background compaction. Crash-safe ACID transactions.
  • HNSW Index — C++ approximate nearest-neighbor graph implementing the full Malkov & Yashunin 2018 algorithm. GIL-free search, persistent thread pool, AVX2 batch scoring.
  • Query Engine — 5-step pipeline: metadata filter pushdown (O(1)) → ANN retrieval → candidate merge → LSM fetch → hybrid scoring (70% cosine + 30% Okapi BM25).

Optional scale layers: Product Quantization (4–32× memory compression) and IVF coarse index (Voronoi-cell partitioning for 10M+ vector corpora).

Unlike hosted solutions (Pinecone, Weaviate), you own the whole stack — storage format, index, scoring, and server.

Dependencies: Core requires only numpy. The optional REST API server (pip install akshamdb[server]) additionally needs Flask. No sklearn/scipy in the core path — the BM25 keyword index is implemented from scratch.


Architecture

┌──────────────────────────────────────────┐
│          REST API  (Flask)               │
│   akshamdb-server / python -m akshamdb   │
└─────────────────┬────────────────────────┘
                  │
┌─────────────────▼────────────────────────┐
│             AkshamDB  (public API)       │
│ akshamdb.open() → add / search / delete  │
└─────────────────┬────────────────────────┘
                  │
┌─────────────────▼────────────────────────┐
│            VectorDBEngine                │
│  ┌───────────┐  ┌────────┐  ┌─────────┐ │
│  │  LSMStore │  │  HNSW  │  │ Query   │ │
│  │  (Python) │  │ (C++)  │  │ Engine  │ │
│  └───────────┘  └────────┘  └─────────┘ │
│  ┌───────────┐  ┌────────┐              │
│  │    PQ     │  │  IVF   │  (optional)  │
│  │(compress) │  │(coarse)│              │
│  └───────────┘  └────────┘              │
└──────────────────────────────────────────┘
      │               │            │
 [WAL + SSTables]  [index.bin]  [manifest.json]
 [Bloom filters]   [pq_*.pkl]   [ivf_index.pkl]

What Has Been Built

Phase 1 — Foundation

Component Details
LSM Store WAL append (with fsync for crash durability) → in-memory OrderedDict (memtable) → SSTable files on flush. Atomic WAL truncation after each flush. Background compaction thread merges SSTables with atomic rename. Thread-safe SSTable ID allocation. Bloom filter per SSTable skips disk on definite misses. Full ACID: begin / commit / rollback.
HNSW (C++) Multi-layer navigable small-world graph implementing Malkov & Yashunin 2018. Flat contiguous vector storage. Format v3 serialization (4-byte HNSW magic header + version + full edge lists). Supports reading legacy v1/v2 files. Exposed to Python via pybind11 with GIL released during search().
Query Engine Five-step pipeline: (1) filter pushdown on in-memory metadata index, (2) ANN retrieval via HNSW, (3) intersect filter ∩ ANN candidates, (4) fetch docs from LSM, (5) hybrid cosine + Okapi BM25 scoring (alpha=0.7 vector, 0.3 BM25).
Embedder all-mpnet-base-v2 via Sentence-Transformers. 768-dim float32 vectors.
Reranker cross-encoder/ms-marco-MiniLM-L-6-v2 CrossEncoder for post-search reranking.
Chunker ingestion/chunker.py — sentence, paragraph, and word-based strategies with configurable chunk size (default 512 chars) and overlap (default 128 chars).
REST API Optional Flask server (pip install akshamdb[server]). Start with akshamdb-server or python -m akshamdb.server. Insert, search, batch search, get, delete, transactions, health, stats endpoints.

Phase 2 — Parallelism & Throughput

Feature Details
GIL-free Search C++ HNSW::search() releases the GIL — concurrent Python threads run in native code without waiting on each other.
Persistent Thread Pool C++ thread pool started at engine init. Reused across calls, no per-query spawn overhead. Controlled by Config.PARALLEL_SEARCH_WORKERS. Explicit destructor prevents thread leak on shutdown.
Batch Search (search_batch_reuse) Phase 1: all query traversals run in the C++ thread pool in parallel. Phase 2: build a union of candidates and score them with an AVX2 blocked kernel — each candidate vector loaded exactly once for all queries. Python-layer scoring is vectorised via numpy batch matmul. Typical 3–10× throughput over a Python loop of single searches, depending on corpus size and batch size.
Python-level Parallel Fallback search_batch_parallel uses ThreadPoolExecutor as a fallback when search_batch_reuse is unavailable. GIL-free C++ search lets threads genuinely overlap.

Phase 3 — Scale & Reliability

Feature Details
Three-tier Startup Tier 1 (Fast): saved HNSW node count matches LSM doc count → load index + remap IDs only. Tier 2 (Incremental): saved HNSW has fewer nodes → load index + add only delta docs from WAL. Tier 3 (Full rebuild): no saved index. Startup cost is O(delta), not O(N).
Product Quantization (PQ) Subspace k-means codebooks (no sklearn dependency). train → encode → adc_distances pipeline. Asymmetric Distance Computation replaces exact cosine when active. Codebooks and codes persist to disk (pq_codebooks.pkl, pq_codes.npy). Enable when memory > ~50 GB.
IVF Coarse Index Partitions corpus into K Voronoi cells via k-means. Search probes only nprobe nearest cells. Supports IVF+PQ mode (ADC scoring inside lists). Enable when HNSW latency is too high at 10M+ vectors.
Profile-Driven Optimization tools/profile_hotspots.py — cProfile analysis of the full insert + search pipeline. Reports time breakdown by subsystem (HNSW graph build, WAL/JSON, LSM/SSTable, threading, Python overhead), hottest functions, WAL microbenchmarks. Includes a _build_recommendations() engine that rates bottlenecks HIGH / MEDIUM / INFO and prints concrete action items.
Runtime Index Tuning index.set_ef_search(n) tunes HNSW search quality at runtime without rebuilding the graph. Takes effect on the next search call.
pip-installable Package setup.py wires CMake into pip's build process. pip install -e . configures and compiles the C++ extension automatically. akshamdb-server CLI entry point registered via pyproject.toml.
Bloom Filters Per-SSTable Bloom filter saved alongside each .bin file. Skips disk I/O for definite key misses during reads.
Ghost Node Handling On rollback, id_map entries are purged so rolled-back docs are invisible to all future searches. HNSW graph nodes can't be removed — they stay as ghost nodes. On close(), the index file is not overwritten if ghosts exist, so the last clean save is preserved for fast-path startup.

Phase 5 — Correctness & Performance Fixes (v0.3.0)

Fix Details
Real Okapi BM25 The keyword scorer was previously sklearn TF-IDF cosine similarity. It is now a pure-Python Okapi BM25 implementation (k1=1.5, b=0.75) backed by an inverted index. Only documents containing at least one query term are scored, making retrieval sub-linear in corpus size. No sklearn dependency in the core path.
search_batch O(N²) fix Previously, each query in a batch called cosine_similarity(query_vec, full_matrix) individually — O(M×N×V) with poor BLAS utilisation. Real BM25 with an inverted index eliminates the dense matrix multiply entirely. Python-layer vector scoring is now a single numpy batch matmul instead of per-doc cosine loops.
Vectorised candidate scoring _hybrid_score_and_rank now stacks all candidate vectors into a matrix and computes all cosine similarities in one doc_mat @ query call, replacing N individual np.dot calls.
print() → logging All print() calls in db_engine.py, lsm_store.py, and ivf_index.py replaced with logging.getLogger(__name__). Control verbosity via Python's standard logging config.
sklearn removed from core deps scikit-learn is no longer a required dependency. Core akshamdb only needs numpy.

Phase 4 — Correctness & Algorithm Fixes (v0.2.0)

Fix Details
Real HNSW Beam Search search_layer now uses two priority queues (Malkov & Yashunin 2018, Algorithm 2): C (min-heap of candidates to expand) and W (max-heap of current best ef neighbors). Early termination fires when the closest unexplored candidate is farther than the worst node in W, giving O(ef · log(ef) · M) complexity vs the previous O(N) BFS.
SELECT-NEIGHBORS-HEURISTIC Neighbor selection during add_point now uses Algorithm 4 from the paper. The heuristic rejects a candidate c if an already-selected neighbor r satisfies dist(c, r) < dist(c, query) — meaning r already "covers" the direction toward c. This prevents graph connectivity from collapsing into one region of space and keeps recall high at all ef values. Falls back to filling from remaining candidates if the heuristic is too aggressive.
Neighbor Degree Cap mmax0 = 2M at layer 0, mmax = M at upper layers, matching the paper exactly. Over-degree neighbors are pruned using the same heuristic, not a simple truncation.
Thread-safe RNG get_random_level() uses thread_local std::mt19937 seeded from std::random_device{}() — each thread gets its own generator, eliminating lock contention on the level-assignment path.
Format v3 Serialization Index files now begin with a 4-byte magic HNSW header followed by an integer version field. This makes format detection unambiguous — the old v2 sentinel (FORMAT_VERSION=2 as the first int) collided with dim=2 in v1 files. v1 and v2 files are still readable on load.
WAL Fsync Durability _write_wal() calls f.flush() + os.fsync(f.fileno()) after every write. WAL truncation after flush uses an atomic rename (wal.log.trunc → wal.log) to prevent partial truncation on crash.
LSM Thread-safe SST IDs _alloc_sst_id() now acquires _sst_lock internally so concurrent compaction threads can't race to allocate the same SSTable ID.
HybridSearch In-place Upsert add_documents() checks _doc_id_to_idx before appending. If the same doc_id is added twice, the text is updated in-place at the existing slot instead of creating a ghost entry. The O(1) swap-remove in remove_document() and O(1) update in update_document() are correct in all cases.
Delete Syncs BM25 delete() now calls _bm25.remove_document() and syncs query_engine.hybrid_search so deleted documents don't appear in keyword results after removal.
Parameter Clamping HNSW constructor clamps M, ef_construction, and ef_search to ≥ 1 so an invalid configuration can never cause an empty-heap crash in search_layer.
Dead Code Removed Duplicate Metrics class removed from utils/schema.py (the real one lives in utils/metrics.py). Dead distance(vector, vector) private method removed from C++ HNSW (was never called).

Installation

Prerequisites: Python 3.8+, CMake 3.15+, GCC/Clang with C++14 support

Option A — install from wheel (fastest)

python3 -m venv venv
source venv/bin/activate
pip install akshamdb-0.1.0-cp310-cp310-linux_x86_64.whl

setup.py wires the CMake build into pip, so a single command configures, compiles, and installs the C++ extension:

python3 -m venv venv
source venv/bin/activate

# Core only (no embedding model, no server)
pip install -e .

# With REST API server
pip install -e ".[server]"

# Development (pytest, benchmarks)
pip install -e ".[dev]"

Option C — manual build

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

cd cpp && mkdir build && cd build
cmake .. -Dpybind11_DIR=$(python -m pybind11 --cmakedir) -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
cp hnsw_cpp*.so ../../akshamdb/
cd ../../

Rebuild wheel from source

rm -f akshamdb/hnsw_cpp*.so
python3 -m build --wheel
# wheel lands at dist/akshamdb-0.1.0-cp310-cp310-linux_x86_64.whl

Use python -m build (PEP 517), not python setup.py bdist_wheel directly — the latter bypasses pyproject.toml's [project] metadata and silently produces a broken akshamdb-0.0.0 wheel instead.

Verify

python3 -c "import akshamdb; print(akshamdb.__version__)"

Quick Start (Python API)

import akshamdb

# Open (or create) a database
db = akshamdb.open("./my_db", dim=768)

# Insert a single document
db.add("doc_1", vector, text="Hello world", metadata={"source": "wiki"})

# Insert many at once (faster than a loop)
db.add_many(ids, vectors, texts=texts, metadata=meta_list)

# Search — returns [{"id", "score", "text", "metadata"}, ...]
results = db.search(query_vec, top_k=5)

# Filtered search — metadata keys are ANDed
results = db.search(query_vec, top_k=5, filter={"source": "wiki"})

# Hybrid search — blends vector cosine (70%) with BM25 (30%)
results = db.search(query_vec, top_k=5, text="what is deep learning")

# Batch search — parallel ANN, much faster than N individual searches
batch = db.search_batch([vec1, vec2, vec3], top_k=5)

# Fetch and delete
doc = db.get("doc_1")
db.delete("doc_1")

# Check membership
if "doc_1" in db:
    print("exists")

# Stats
print(db.info())  # documents, index_nodes, data_path, performance

db.close()

Context Manager

with akshamdb.open("./my_db", dim=768) as db:
    db.add("doc_1", vector)
    results = db.search(query_vec, top_k=5)
# db.close() called automatically

Transactions

with db.transaction():
    db.add_many(ids, vectors, texts=texts)
# commits on clean exit, rolls back on any exception

Ingest throughput note: Outside a transaction, add_many() saves the HNSW index to disk after every call (crash-safe default). This limits throughput to roughly 100–200 docs/s for 768-dim vectors on a typical SSD. Wrapping bulk ingestion in with db.transaction() batches all writes into one WAL flush and one index save, achieving 800–1200+ docs/s. For multi-million-document corpora, always use transactions.

Logging

AkshamDB uses standard Python logging — no output is printed unless you configure a handler:

import logging
logging.basicConfig(level=logging.INFO)   # show startup and rebuild messages
# or silence everything:
logging.getLogger("akshamdb").setLevel(logging.WARNING)

With Embedding

from embedding.embedder import Embedder
import akshamdb

embedder = Embedder()   # loads all-mpnet-base-v2

with akshamdb.open("./my_db", dim=768) as db:
    texts = ["Machine learning is a subset of AI.", "Neural networks learn from data."]
    vecs  = embedder.encode(texts)
    db.add_many(["doc_0", "doc_1"], list(vecs), texts=texts, metadata=[{"src": "wiki"}] * 2)

    q_vec = embedder.encode(["What is deep learning?"])[0]
    for r in db.search(q_vec, top_k=3, text="deep learning"):
        print(f"{r['score']:.3f}  {r['text']}")

With Chunking + Ingestion

from ingestion.loader import prepare_documents
from embedding.embedder import Embedder
import akshamdb

raw_docs = ["Long document text here ...", "Another document ..."]
chunks, ids, metadata = prepare_documents(raw_docs)

embedder = Embedder()
vecs = embedder.encode(chunks)

with akshamdb.open("./my_db", dim=768) as db:
    db.add_many(ids, list(vecs), texts=chunks, metadata=metadata)

REST API

Start the Server

# Default: port 8000, data at ./data
akshamdb-server

# Custom settings via env vars
AKSHAMDB_PATH=/my/db AKSHAMDB_DIM=768 AKSHAMDB_PORT=9000 AKSHAMDB_TOP_K=10 akshamdb-server

# Or: python -m akshamdb.server

# Production (gunicorn, 4 workers)
gunicorn -w 4 -b 0.0.0.0:8000 "akshamdb.server:create_app()"

Endpoints

Method Endpoint Description
POST /documents Insert one or many documents
GET /documents/<id> Fetch document (?include_vector=true for raw vector)
DELETE /documents/<id> Delete a document
POST /search Single search — hybrid vector + BM25
POST /search/batch Batch search — parallel ANN
GET /health Live count: documents + index nodes
GET /stats Full performance metrics

Examples

# Insert (text auto-embedded if sentence-transformers is installed)
curl -X POST http://localhost:8000/documents \
  -H "Content-Type: application/json" \
  -d '{"documents": [
        {"text": "Machine learning is cool", "metadata": {"source": "wiki"}},
        {"id": "my-id", "vector": [0.1, 0.2, ...], "text": "raw vector insert"}
      ]}'

# Search by text query
curl -X POST http://localhost:8000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "deep learning", "top_k": 5}'

# Search with metadata filter
curl -X POST http://localhost:8000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "AI", "top_k": 5, "filters": {"source": "wiki"}}'

# Batch search
curl -X POST http://localhost:8000/search/batch \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"query": "deep learning"}, {"query": "neural nets"}], "top_k": 3}'

# Health check
curl http://localhost:8000/health

# Stats
curl http://localhost:8000/stats

Configuration

All flags live in utils/schema.py → Config:

class Config:
    # Storage
    DATA_PATH         = "data"
    MEMTABLE_LIMIT    = 1000   # flush to SSTable after N documents in memory
    COMPACT_THRESHOLD = 4      # merge SSTables when this many accumulate
    BATCH_SIZE        = 1000

    # HNSW Index
    VECTOR_DIM            = 768   # all-mpnet-base-v2
    HNSW_M                = 16    # graph connections per node (16 for production recall)
    HNSW_EF_CONSTRUCTION  = 200   # build quality (higher = better recall, slower inserts)
    HNSW_EF_SEARCH        = 150   # candidates explored per query (150 for high recall)

    # Query
    DEFAULT_TOP_K     = 5
    ENABLE_RERANKING  = True
    FILTER_PUSHDOWN   = True

    # Parallelism
    ENABLE_METRICS          = True
    ENABLE_PARALLEL_SEARCH  = True    # GIL-free per-query parallel scoring
    PARALLEL_SEARCH_WORKERS = None    # None = os.cpu_count()
    ENABLE_BATCH_REUSE      = True    # AVX2 union-of-candidates batch path

    # Product Quantization — enable when memory > ~50 GB
    ENABLE_PQ = False
    PQ_M      = 96     # subspaces (must divide VECTOR_DIM evenly)
    PQ_K      = 256    # centroids per subspace (uint8 → max 256)

    # IVF Coarse Index — enable when corpus > ~10M vectors
    ENABLE_IVF     = False
    IVF_N_CLUSTERS = 256   # Voronoi cells — rule of thumb: sqrt(corpus_size)
    IVF_NPROBE     = 8     # cells probed at search time

Tuning cheat sheet:

Situation Setting
Need better recall Raise HNSW_EF_SEARCH (150+) or HNSW_M (16–32)
Slow inserts Lower HNSW_EF_CONSTRUCTION (100) — small recall cost
Memory > 50 GB ENABLE_PQ = True — 4–32× compression, ~5% recall loss
Corpus > 10M vectors ENABLE_IVF = True, IVF_N_CLUSTERS = int(sqrt(N))
Recall too low with IVF Raise IVF_NPROBE or disable IVF and use HNSW only
Runtime ef tuning db._engine.index.set_ef_search(200) — no rebuild needed

Benchmarking

All tools are in tools/. Data directories are generated at runtime and are not committed.

Script What it does
tools/e2e_benchmark.py Core benchmark function: insert N docs, run M queries, report insert throughput and search latency (avg / p50 / p99). Optional Recall@K vs brute-force.
tools/run_bench_suite.py Runs four preset scenarios: large 20K (vector-only), hybrid 5K (text+vector), HNSW m8/ef20, HNSW m16/ef50.
tools/compare.py Head-to-head: AkshamDB vs FAISS vs ChromaDB — insert throughput, search p50/p99, memory delta, Recall@K, recovery time.
tools/profile_hotspots.py cProfile hotspot analysis: time breakdown by subsystem (HNSW, WAL, LSM, threading), top hot functions, WAL microbenchmarks, and a recommendation engine that rates bottlenecks HIGH / MEDIUM / INFO with specific action items.
tools/tune_parallel_search.py Sweep over thread-pool sizes and ef values. Writes a JSON report of QPS vs latency trade-offs.
tools/cleanup_baseline_benchmarks.py Deletes old baseline_benchmark_YYYYMMDD_HHMMSS/ dirs, keeping the newest N.
run_baseline_benchmark.py Quick baseline: python run_baseline_benchmark.py <n_docs> <n_iters>

Run the full suite

cd tools
python run_bench_suite.py
# results saved to benchmark_results/

Profile hotspots

python tools/profile_hotspots.py

Compare against FAISS / ChromaDB

pip install faiss-cpu chromadb          # optional — skipped gracefully if missing

python tools/compare.py                                  # 10K vectors, dim=128
python tools/compare.py --n 50000 --dim 256 --queries 500
python tools/compare.py --n 10000 --skip-chroma --out results/bench.json
python tools/tune_parallel_search.py             # quick sweep (n=5000)
python tools/tune_parallel_search.py --n 20000   # larger sweep

Testing

python3 test_core.py                              # quick smoke test
python3 -m pytest tests/test_engine.py -v        # unit + integration suite
python3 -m pytest tests/test_reliability.py -v   # crash recovery and WAL tests
python3 -m pytest tests/test_benchmark_integration.py -v   # benchmark integration
python3 run_baseline_benchmark.py 1000 10        # 1000 docs, 10 iterations

Data Layout

After the database is created at <data_path>/:

<data_path>/
  wal.log           — Write-Ahead Log (append-only, JSON lines, fsynced after each write)
  manifest.json     — ordered list of active SSTable files
  sst_1.bin         — SSTable (pickled dict of documents)
  sst_1.bloom       — Bloom filter for sst_1.bin
  ...
  index.bin         — HNSW graph (C++ binary format, v3 serialization with magic header)
  pq_codebooks.pkl  — PQ subspace codebooks (if ENABLE_PQ)
  pq_codes.npy      — encoded corpus codes, shape (N, M) uint8 (if ENABLE_PQ)
  pq_hnsw_ids.npy   — hnsw_id per row in pq_codes.npy (if ENABLE_PQ)
  ivf_index.pkl     — IVF inverted lists and centroids (if ENABLE_IVF)

Index format compatibility:

  • v3 (current): 4-byte magic HNSW + int version + data. Unambiguous detection.
  • v2 (readable): FORMAT_VERSION=2 sentinel as first int.
  • v1 (readable): raw dim as first int. Edge lists not saved — requires full rebuild on load.

Troubleshooting

C++ extension not found

cd cpp && rm -rf build && mkdir build && cd build
cmake .. -Dpybind11_DIR=$(python -m pybind11 --cmakedir) -DCMAKE_BUILD_TYPE=Release
make -j$(nproc) && cp hnsw_cpp*.so ../../akshamdb/

Data not persisting — always call db.close() or use the context manager:

with akshamdb.open("./data", dim=768) as db:
    db.add(...)
# close() is called automatically

Slow search — raise ef_search at runtime without rebuilding:

db._engine.index.set_ef_search(200)
# or permanently in Config:
Config.HNSW_EF_SEARCH = 200
Config.HNSW_M         = 16

High memory usage — enable PQ compression:

Config.ENABLE_PQ = True
# Codebooks are trained automatically on the next full rebuild

Search recall too low — raise IVF nprobe or disable IVF:

Config.IVF_NPROBE = 32   # probe more cells
# or
Config.ENABLE_IVF = False  # fall back to full HNSW scan

Old index file fails to load — v3 reader handles v1/v2/v3 automatically. If you see a load error, delete index.bin and let it rebuild from LSM on next startup.


Changelog

v0.3.0 (2026-06-24)

  • [Fix] Real Okapi BM25 — keyword scorer replaced: TF-IDF cosine similarity (sklearn) → Okapi BM25 with an inverted index (k1=1.5, b=0.75, stdlib only). Scoring is now sub-linear in corpus size.
  • [Fix] search_batch O(N²) — batch BM25 no longer runs dense matrix multiplies per query. Real BM25 inverted index eliminates the bottleneck. Python scoring path vectorised with numpy batch matmul.
  • [Fix] Vectorised candidate scoring — _hybrid_score_and_rank uses a single doc_mat @ query call instead of per-doc np.dot loops.
  • [Fix] print() → logging — all lifecycle/rebuild messages in db_engine.py, lsm_store.py, ivf_index.py use logging.getLogger(__name__). No output unless you configure a handler.
  • [Fix] sklearn removed from core — scikit-learn is no longer a required dependency. Core install needs only numpy.
  • [Docs] Ingest throughput clarified — transaction requirement for high throughput documented explicitly.
  • [Docs] REST API (Flask server) and optional features (PQ, IVF) documented.

v0.2.0 (2026-06-19)

  • [Fix] Real HNSW beam search — dual priority-queue implementation (Algorithm 2, Malkov & Yashunin 2018). O(ef·log(ef)·M) vs previous O(N) BFS.
  • [Fix] SELECT-NEIGHBORS-HEURISTIC — neighbor selection during insert now uses Algorithm 4. Diverse neighbor sets keep the graph navigable across all regions of space.
  • [Fix] Neighbor degree cap — mmax0 = 2M at layer 0, mmax = M at upper layers. Over-degree lists pruned with the heuristic, not a truncation.
  • [Fix] Thread-safe RNG — get_random_level() uses thread_local mt19937 seeded per thread from random_device. No mutex needed on the level-assignment path.
  • [Fix] Index format v3 — HNSW magic prefix + version field eliminates format-detection ambiguity. v1/v2 files still load correctly.
  • [Fix] WAL fsync — _write_wal() calls fsync after every entry. Truncation uses atomic rename.
  • [Fix] LSM SST ID thread safety — _alloc_sst_id() holds _sst_lock internally.
  • [Fix] HybridSearch duplicate doc_id — add_documents() updates in-place on duplicate; no zombie BM25 entries.
  • [Fix] Delete syncs BM25 — deleted docs removed from keyword index and query engine state.
  • [Fix] HNSW destructor — explicit ~HNSW() releases the thread pool, no resource leak on shutdown.
  • [Fix] Parameter clamping — M, ef_construction, ef_search clamped to ≥ 1.
  • [Remove] Dead code — duplicate Metrics class in schema.py; dead distance(vector, vector) in C++ HNSW.
  • [Config] HNSW_M default raised from 8 → 16 for production-quality graph connectivity.
  • [Config] HNSW_EF_SEARCH default raised from 50 → 150 for high recall on real corpora.

v0.1.0

  • Initial release: LSM store, HNSW index, query engine, hybrid BM25+vector scoring, PQ, IVF, batch search, REST API.

Built With

  • C++ + pybind11 — HNSW graph, AVX2 distance kernel, persistent thread pool, GIL-free search
  • Python — LSM store, query engine, hybrid search, ingestion pipeline, REST API
  • Sentence-Transformers — all-mpnet-base-v2 embedding model (768-dim)
  • CrossEncoder — ms-marco-MiniLM-L-6-v2 reranker
  • Flask — REST API server
  • NumPy / scikit-learn — vector ops, TF-IDF BM25 scoring, PQ k-means

Metadata

Release files for akshamdb 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for akshamdb 0.1.0
File Size Uploaded Provenance Actions
akshamdb-0.1.0.tar.gz 266.4 kB Without provenance Details for akshamdb-0.1.0.tar.gz

Release files / akshamdb-0.1.0.tar.gz

Download URL akshamdb-0.1.0.tar.gz
Size 266.4 kB
Tags Source
SHA-256 checksum
How to use checksums
741758e8b2f19279a6fe54c1334110a54354f47f055cc920c927442d955623ed
BLAKE2b-256 checksum
How to use checksums
e48cf1ba1eb6e1b3cd9ec93d0dc499c848954b48e8b5812cfbc32e1523b12718
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.12

Security for akshamdb 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page