esp32kb

Build an on-device knowledge base for the KB ESP32-P4 demo — a MiniLM-L3 embedding search that runs entirely on the board, with no PC attached at runtime.

Point the tool at a folder of markdown/text documents and it emits everything the firmware flashes: sentence-aware chunks, INT8 embeddings, the packed kb.img container (vectors + text + file index + WordPiece vocabulary) and the tiny generated headers the sketch compiles against.

Install

uvx esp32kb --help            # or: pip install esp32kb
# from a checkout:
uv run --project esp32kb esp32kb --help

Usage

# 1. documents -> kb.img + generated headers
esp32kb embed --root path/to/documents --sketch-dir path/to/Arduino/project

# 2. print the arduino-cli command that flashes app + model + kb.img
esp32kb flash --sketch-dir path/to/Arduino/project   # then answer "y" to run it

Everything defaults relative to --sketch-dir (the current folder by default), so the demo ships with its MiniLM-L3 model and one flag is enough. esp32kb flash also writes the bundled partitions.csv into the sketch folder if it is missing.

esp32kb embed

esp32kb flash

The sequence length and embedding width are read straight from the ONNX model, so the chunks, the INT8 vectors and the generated kb_config.h the firmware compiles against can never disagree with the model. No flags to keep in sync.

Pipeline

documents
  -> chunks (<= seq-2 WordPiece tokens each)
  -> MiniLM-L3 embeddings (masked mean pool, L2 normalized)
  -> symmetric INT8 quantization, one global scale
  -> vectors.bin / meta.bin / chunks.bin / files.bin / vocab.txt
  -> kb.img        single little-endian container with a section table
  -> kb_config.h   KB_SEQ / KB_DIM / KB_TOPK + container magics (C++ constants)
  -> kb_queries.h  KB_EXAMPLES[] example query texts for the sketch

The vocabulary travels inside kb.img, so the device tokenizes queries itself.

Additive attention mask

If the ONNX model exposes a second attention_mask input (the variant built by model/export_onnx_mask.py), esp32kb feeds it the correct per-query padding bias (0 on real tokens, negative on [PAD]). This preserves retrieval quality at longer sequence lengths, where the frozen all-ones mask used by the plain export otherwise collapses query embeddings. With the stock single-input ONNX the behavior is unchanged.

Outputs