laghu

model2vec makes static embedding models: models that turn a text into a vector by looking up a stored vector for each of its tokens and averaging them. There is no neural network to run, so they are very fast, and the whole model is two files: an embedding table with one row per token, and a tokenizer that splits text into those tokens. laghu (लघु, Sanskrit for “light”) loads the same models and gives the same vectors with far less memory. A model loaded with laghu has model2vec’s encode, encode_as_sequence and tokenize, and its output matches model2vec’s bit for bit. The one difference, for an empty list of texts, is listed at the end of this page.

The memory goes on two things. model2vec reads the whole embedding table into each process; for potion-multilingual-128M that is 512 MB. That model’s tokenizer is a Unigram tokenizer, which splits text by choosing the most likely sequence of pieces from a list of 500,353, and the tokenizers library keeps a search structure (a character trie) over those pieces that takes about another 610 MB. After encoding, a model2vec process holds about 1.3 GB of private memory, memory that belongs to that process alone and cannot be shared.

laghu avoids both copies. It maps the table from model.safetensors into memory read-only, so the operating system loads pages of the file only when they are read, keeps them in its page cache, shares them between every process that maps the same file, and can drop them again when memory is short. For a Unigram model it also replaces the tokenizer’s trie: it builds a vocabulary file once per tokenizer.json, holding the pieces in a compact marisa trie with their ids and scores, which every process maps and shares in the same way, and it segments text with a Viterbi search written in Python, or in Rust when the optional laghu-rs wheel is installed. The text normalization and the first split into words still come from tokenizers, built from the same tokenizer.json with the vocabulary left out. The same model then takes about 60 MB of private memory per process, most of it the Python interpreter and its imports.

Models whose tokenizer is WordPiece, BPE or word-level keep using tokenizers for tokenizing. laghu still maps their tables, so how much they save depends on how large the table is compared with the tokenizer.

Install

uv add laghu        # or: pip install laghu

model2vec is not a dependency. Install the extra, uv add 'laghu[fallback]', if you want load to fall back to model2vec when laghu cannot read a model, or want to use same_as_model2vec. The extra installs model2vec 0.9, which laghu matches. model2vec 0.10 gives different vectors, and its tokenize no longer takes max_length.

laghu has an optional Rust core, the wheel laghu-rs (module laghu_rs, Python 3.11 or later). Install it with pip install laghu-rs, or with laghu’s extra, uv add 'laghu[rust]', from laghu 0.3.0.

The optional Rust core

With laghu-rs installed, laghu tokenizes Unigram models in Rust, on all cores, with the GIL released. laghu_rs normalizes, splits out added tokens and pre-tokenizes with the tokenizers crate, then runs laghu’s Viterbi search over a byte trie in the shared vocabulary file. For a float32 table without weights or a token mapping, it also averages the vectors. The ids and vectors are the same as the pure Python path’s and model2vec’s, bit for bit.

potion-multilingual-128M encodes 500-character chunks at 39,750 chunks/s with laghu-rs on the 14 cores of an M3 Max. The same model encodes 1,736 chunks/s in pure Python and 5,276 chunks/s with model2vec 0.9. The process holds 64 MB of private memory after encoding with laghu-rs.

laghu tokenizes in pure Python when any of these is true:

• laghu_rs cannot be imported.
• LAGHU_RUST=0 is set when the model is loaded.
• The model is loaded with shared=False, or its tokenizer.json truncates or pads.

laghu never uses laghu-rs for a model it tokenizes with tokenizers: WordPiece, BPE and WordLevel models, and Unigram models that use a feature laghu does not reproduce.

laghu splits text in Python and segments it in Rust when any of these is true:

• laghu_rs.TOKENIZERS, the tokenizers crate version laghu-rs was built with, differs from the installed Python tokenizers. laghu-rs 0.1.0 is built with 0.23.2.
• The tokenizers crate cannot read the tokenizer.json.
• The crate splits any text in PROBE, 46 of laghu’s built-in difficult texts, differently from laghu’s Python split. laghu logs a warning.

The ids are the same on every path.

Use

from laghu import load

m = load('minishlab/potion-multilingual-128M')      # a folder, or a Hugging Face repo id (the local cache is tried first)
v = m.encode(['The quick brown fox.', 'कर्मण्येवाधिकारस्ते'])   # float32, (2, 256), unit rows: the same array model2vec returns
m.tokenize(['hello world'])
m.encode_as_sequence('hello world').shape          # (n_tokens, 256)

(2, 256)

load takes a local folder or a Hugging Face repo id, and returns a LeanModel. A repo already in the local Hugging Face cache loads without the network. If laghu cannot load the model, load logs why and tries model2vec’s StaticModel instead, which needs laghu[fallback]; if that fails too, it raises laghu’s original error. Pass fallback=False to get the error without trying model2vec. Neither loader can read tables stored as bfloat16 or float8, because numpy has no such types.

The vocabulary file for a Unigram model is kept in $LAGHU_CACHE, else $XDG_CACHE_HOME/laghu, else ~/.cache/laghu, and next to tokenizer.json when the cache cannot be written. The first load builds it in a second or two; later loads, in any process, map it in a few hundredths of a second. If tokenizer.json changes or the file is damaged, it is built again. load(..., shared=False) keeps the vocabulary in a Python dict instead, which costs about 80 MB more per process.

To check a model of your own, same_as_model2vec(path) loads it both ways and compares the results on laghu’s built-in test texts, or on texts you pass:

from laghu import same_as_model2vec
same_as_model2vec('minishlab/potion-base-8M', ['my', 'own texts'])

/Users/71293/code/personal/orgs/laghu/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

{'lean': False,
 'n': 2,
 'ids': True,
 'dtype': True,
 'f32': True,
 'f16': True,
 'exact': True,
 'seq': True,
 'median': True,
 'unk': True,
 'tokens': True}

The result says, for each check, whether the two loaders agreed: the token ids (ids), the vectors cast to float32 (f32) and to float16 (f16), the exact dtype (dtype) and bits (exact), encode_as_sequence (seq), median_token_length (median), the unknown token id (unk) and the token list (tokens). lean says whether laghu used its own Unigram tokenizer, and n is the number of texts.

What matches, and what it costs

Every model below gives model2vec’s ids and vectors, bit for bit, on 4,726 texts. The texts include prose in 36 languages and random strings from 30 Unicode blocks, and also control characters, special tokens, emoji sequences, combining marks, bidirectional text controls and a 200,000-character string. Each memory figure comes from a fresh process. On macOS it is phys_footprint after encoding, and lifetime_max_phys_footprint for the peak. On Linux the same columns would hold RssAnon and the peak resident memory VmHWM. Speed is encode over 3,226 texts on an M3 Max. The “laghu tokenizes with” column says whether laghu used its Python Viterbi or tokenizers. The table was measured without laghu-rs.

+map marks a vocabulary-quantized model, which stores fewer rows than it has tokens and a mapping from each token to its row. +weights marks a model that scales each token’s vector by a stored weight. The potion-multilingual-128M row was measured again on 2026-10-13 with the shared vocabulary file. The other Unigram rows were measured with the vocabulary in a dict, which cost potion-multilingual-128M about 80 MB more private memory per process.

Without laghu-rs, laghu is slower for Unigram models, at between about a third of model2vec’s rate and about the same. tokenizers spreads a batch over all cores, while the Python Viterbi runs on one. With laghu-rs, laghu also runs on all cores and is faster than model2vec on potion-multilingual-128M: 39,750 against 5,276 chunks/s, as measured in the optional Rust core. The other Unigram models have not been measured with laghu-rs. These are batch measurements and say nothing about the time to encode a single query.

laghu keeps a cache of words it has already segmented. It helps when the tokenizer first splits the text into words, as jina-v3, bge-m3 and multilingual-e5 do: natural text runs 1.6 to 1.9 times faster on a first pass and 3 to 5 times faster when the same text comes again. The bench page records these measurements and why two faster Rust tokenizers, tokie and gigatoken, were not adopted.

For the other models both loaders tokenize with the same tokenizers code. Their timing samples last about 0.1 seconds and vary by up to half between runs, so the differences in speed there mean nothing.

What is not supported

• Quantizing or truncating a model while loading it (model2vec’s quantize_to, dimensionality and vocabulary_quantization). Each would make a new copy of the table, which is what laghu exists to avoid. Load with model2vec if you need them.
• A Unigram model with single_word added tokens is tokenized with tokenizers, though its table is still mapped. tokenizers decides where a word ends with Unicode’s Alphabetic property, and Python’s str.isalpha does not match it.
• use_multiprocessing is accepted and ignored; the result is the same.
• encode([]) returns an empty array of shape (0, dim), where model2vec raises.

Developing

laghu uses nbdev 3 with hatchling: the notebooks in nbs/ are the source. uv run nbdev-prepare exports the package, runs the tests and rebuilds this README. uv run nbdev-test --flags slow also runs the benchmark, which downloads about 5 GB of models.

Licence

MIT.