faster-clip

Fast, dependency-light CPU inference for the models a multilingual RAG stack needs — no PyTorch, no transformers, no sentence-transformers, no ONNX. Since 0.2 the compute is a hand-written Rust engine (a single native extension); tokenization uses the tokenizers crate.

pip install faster-clip

Three drop-in replacements, each numerically matching the original HuggingFace pipeline:

Performance

CPU (AMD Ryzen 7 3700X, 8 cores), single item, vs the original torch pipeline:

No torch/transformers install (multi-GB) — one native wheel. Since 0.4 the 2-D weights are stored as int8 (per-output-channel symmetric quantization, upconverted on the fly in the AVX2 kernels); small layernorm/bias params stay f16. Loading copies everything into owned buffers and drops the source mmap, so nothing of the original f32 file stays resident. Measured steady-state RSS on CPU:

Accuracy is essentially unchanged: CLIP cosine ≥ 0.9997 vs torch, SPLADE bit-identical, fill-mask top-5 exact (en + pt).

CLIP embeddings

from faster_clip import ClipModel

clip = ClipModel()  # lazily downloads image / text weights on first use
text_emb = clip.encode_text(["um carro vermelho", "a red car"])   # (2, 512) list
img_emb  = clip.encode_image([open("photo.jpg", "rb").read()])    # (1, 512) list

• encode_text(list_of_str) → list of 512-float vectors.
• encode_image(list_of_image_bytes) → list of 512-float vectors. Pass the raw image bytes (PNG/JPEG/…); decoding + CLIP preprocessing happen in Rust.

Image and multilingual text share the same 512-d space. Normalize yourself for cosine search.

SPLADE sparse encoding (no sparse tensor materialised)

from faster_clip import SparseEncoder
sparse = SparseEncoder()
sparse.encode("banco de dados vetorial")     # -> {'15060': 1.09, '2951': 1.11, ...}

Inference-free: a token→weight lookup, so only the active dims are returned — no dense/sparse vector is ever built. encode, encode_query, encode_document are equivalent; each takes a str or list of str. The weights ship in the wheel.

Fill-mask autocomplete

from faster_clip import FillMask
fm = FillMask()
fm.predict("the weather is very [MASK]", lang="en")
fm.predict("o brasil é um [MASK]", lang="pt")
# -> [{'token_str': 'lixo', 'sequence': 'o brasil é um lixo', 'score': 0.18}, ...]

lang is "en" (bert-base-uncased) or "pt" (neuralmind/bert-base-portuguese-cased).

Model cache

CLIP and BERT weights are downloaded on first use and cached under ~/.cache/faster-clip (override with $FASTER_CLIP_CACHE). Inference releases the GIL, so one instance is safe to share across threads.

License

MIT. Model weights belong to their respective authors (OpenAI / sentence-transformers / neuralmind / Microsoft) under their original licenses.