kaos-nlp-transformers

Part of Kelvin Agentic OS (KAOS) — open agentic infrastructure for legal work, built by 273 Ventures. See the full KAOS package map for the rest of the stack.

kaos-nlp-transformers is the dense-embedding and small-model inference layer for KAOS — a typed Python API over an in-tree Rust cdylib that calls ort (libonnxruntime via Rust) to turn text into float32 vectors and back. It ships a license-vetted model registry and an optional cross-encoder reranker. (Semantic deduplication lives in kaos-content's dedup.levels.semantic.SemanticDedupLevel after KNT-602; install kaos-content[transformers,clustering] for that surface.)

It is dependency-light at the BASE: the install pulls in only numpy, huggingface_hub, and the core KAOS runtime (kaos-core, kaos-nlp-core). No PyTorch, no Python fastembed, no Python onnxruntime — the inference path is a Rust cdylib (kaos_nlp_transformers._rust) shipped inside the wheel; libonnxruntime is statically linked. Both embedding (EmbeddingModel) and cross-encoder reranking (CrossEncoderReranker) run through the same backend on CPU out of the box. Optional extras layer in adjacencies — [model2vec] for the static-numpy lookup backend (~500x CPU speedup) and [mcp] for the MCP tool surface. [gpu] and [openvino] are reserved-and-no-op aliases today (GPU acceleration ships as a separate kaos-nlp-transformers-gpu companion wheel built with --features gpu); [torch] is a deprecated no-op alias scheduled for removal in 0.3.0. Free-threaded Python (3.13t / 3.14t) is supported.

Use, data handling, and authorship disclosure

kaos-nlp-transformers runs inference locally through the bundled Rust cdylib (ONNX Runtime + Rust tokenizers). Once a model is downloaded into the local Hugging Face cache, every subsequent .embed(...) / .rerank(...) call stays in-process — no network, no provider-side data transmission. Model downloads themselves go to the Hugging Face Hub once per (model_id, revision) pair; pin KAOS_NLP_TRANSFORMERS_OFFLINE=1 or HF_HUB_OFFLINE=1 to forbid downloads in production. The SemanticChunker and ExtractiveRanker Programs in this package inherit the same local-only data path. Downstream consumers (notably kaos-llm-core Programs) may transmit text to LLM providers, so callers handling sensitive data should check the consuming package's data-handling disclosure.

This codebase is AI-assisted: substantial portions were generated with Claude (Anthropic) and human-reviewed before commit. Public behavior is covered by the test suite under tests/; live model downloads + GPU paths are opt-in (pytest -m integration and pytest -m gpu). Bug reports welcome via GitHub Issues; security reports follow SECURITY.md.

Install

uv add kaos-nlp-transformers
# or
pip install kaos-nlp-transformers

kaos-nlp-transformers requires Python 3.13 or newer (free-threaded 3.13t / 3.14t supported). The default install is CPU-only via the Rust ort backend. Add the extras you need:

uv add "kaos-nlp-transformers[model2vec]"    # Static-numpy backend (~500x CPU)
uv add "kaos-nlp-transformers[mcp]"          # MCP tool surface
# NVIDIA CUDA acceleration ships as a separate companion wheel:
#   pip install kaos-nlp-transformers-gpu
# Intel OpenVINO is built into a wheel from source:
#   cargo build --release --features openvino
# Semantic deduplication moved to kaos-content (KNT-602):
#   uv add "kaos-content[transformers,clustering]"

0.2.0 migration note (KNT-601). Audit KNT-601 retired the Python fastembed wrapper. Inference now goes through a Rust cdylib (ort + libonnxruntime, statically linked). Same models, same outputs (per-row cosine ≥ 0.9999 vs the prior backend). The EmbeddingModel.load / EmbeddingModel.embed / CrossEncoderReranker public API is unchanged. The [gpu] / [openvino] extras are no-op stubs in 0.2.0a1; the GPU companion wheel ships in 0.2.0a2. The [torch] no-op alias from KNT-501 is still preserved for one more cycle; removed in 0.3.0. The EmbeddingRetriever text-only retriever is deprecated in favor of kaos_content.indexing.SearchableDocument and kaos_content.indexing.SearchableCorpus; removal scheduled for 0.3.0.

Platform coverage: per-platform cp313-abi3 wheels for Linux x86_64 + aarch64 (manylinux + musllinux), macOS aarch64, Windows x86_64 + aarch64. Free-threaded Python (3.13t / 3.14t) loads cleanly — no _check_gil_enabled guard, no py_rust_stemmers SIGSEGV path.

Quick start

import numpy as np
from kaos_nlp_transformers import EmbeddingModel

# Load the v0 default model (BAAI/bge-small-en-v1.5, 33M params, MIT).
# First call downloads and caches; subsequent calls are O(1).
model = EmbeddingModel.load("BAAI/bge-small-en-v1.5")

# Embed a small batch. Returns a float32 numpy array of shape (N, dim).
texts = [
    "Force majeure clauses excuse performance.",
    "Indemnity caps the liability of the seller.",
]
vecs = model.embed(texts)
assert vecs.shape == (2, 384) and vecs.dtype == np.float32

# Cosine similarity over the L2-normalized rows.
def cosine(a, b):
    return float(np.dot(a / np.linalg.norm(a), b / np.linalg.norm(b)))

print(f"sim: {cosine(vecs[0], vecs[1]):.3f}")
# sim: 0.637   (similar legal-contract topic, distinct concepts)

For retrieval over a corpus, build an EmbeddingRetriever:

import asyncio

from kaos_nlp_transformers import EmbeddingRetriever

retriever = EmbeddingRetriever.from_texts(
    texts=[
        "The buyer agrees to mediation in Delaware.",
        "All disputes shall be resolved by arbitration in New York.",
        "Force majeure clauses excuse performance.",
    ],
    doc_ids=[0, 1, 2],
)
hits = asyncio.run(retriever.retrieve("where do contract disputes go?", top_k=2))
for h in hits:
    print(f"{h.score:.3f}  {h.text}")

Phase-8: NLI, NER, and PII

In addition to embedding + reranking, the package ships three small-model inference surfaces for legal-tech and financial document workflows. All three run through the same in-tree Rust ort cdylib — no PyTorch — and emit byte-stable char-offset spans where applicable.

from kaos_nlp_transformers import NliModel, GLiNERExtractor, PiiDetector

# 1) NLI — zero-shot classification via entailment
#    Default: Xenova/nli-deberta-v3-base (Apache-2.0, 184M params)
nli = NliModel.load()
scores = nli.score(
    "Acme Corp shall pay rent of $5,000/month for the leased premises.",
    [
        "This text is about a lease agreement.",
        "This text is about employment.",
    ],
)
for s in scores:
    print(f"  entail={s.entailment:.2f}  neutral={s.neutral:.2f}  contradict={s.contradiction:.2f}")

# 2) GLiNER — zero-shot NER over caller-supplied labels
#    Default: onnx-community/gliner_medium-v2.1 (Apache-2.0, 195M params, 746 MB fp32)
gliner = GLiNERExtractor.load()
[entities] = gliner.extract(
    ["Barack Obama signed the bill on January 1, 2025."],
    labels=["person", "date"],
)
for e in entities:
    print(f"  [{e.score:.2f}] {e.label:<10} {e.text!r}")

# 3) PII — closed-label BERT-small detector (27 MB int8)
#    Default: onnx-community/bert-small-pii-detection-ONNX (Apache-2.0)
#    24 categories: PERSON, EMAIL_ADDRESS, PHONE_NUMBER, US_SSN,
#    CREDIT_CARD, IBAN_CODE, FINANCIAL, LOCATION, ORGANIZATION, ...
pii = PiiDetector.load()
[spans] = pii.detect(["Contact Jennifer Stacey at jen@galera.com today."])
for e in spans:
    print(f"  [{e.score:.2f}] {e.label:<15} {e.text!r}")

When to pick which:

• PiiDetector — fastest (~17× per doc vs GLiNER), use whenever the 24 built-in PII categories cover your need. Redaction, compliance, intake screening.
• GLiNERExtractor — zero-shot over any label set you supply. Slower but flexible — use for domain-specific entities ("indemnification party", "termination notice period") and when accuracy matters more than throughput.
• NliModel — entailment-style classification. Pairs with ZeroShotNLIClassifier in kaos-llm-core for label-set classification without LLM cost.

Concepts

The package is built around a small set of typed primitives.

CLI

kaos-nlp-transformers ships a kaos-nlp-transformers administrative CLI plus a kaos-nlp-transformers-serve MCP server launcher that requires the [mcp] extra:

# Diagnostic envelope: version, registry, device, ONNX providers
kaos-nlp-transformers info --json

# Pre-warm the HF Hub cache with every registered model — useful in
# Dockerfile builds, CI cache-warming, air-gapped image prep.
kaos-nlp-transformers prefetch                  # all 5 families
kaos-nlp-transformers prefetch --include pii    # one family
kaos-nlp-transformers prefetch --model onnx-community/bert-small-pii-detection-ONNX
kaos-nlp-transformers prefetch --dry-run        # show what would be fetched
kaos-nlp-transformers prefetch --quiet --json   # CI-friendly

# stdio MCP server
kaos-nlp-transformers-serve                     # requires [mcp]

Prefetch honors HF_HOME and KAOS_NLP_TRANSFORMERS_CACHE_DIR and exits non-zero on any model failure (continuing through the rest of the batch so one bad row doesn't sink the whole prefetch).

Compatibility & status

Companion packages

kaos-nlp-transformers is one of the packages in the Kelvin Agentic OS. The broader stack:

Packages depend on kaos-core; everything else is opt-in. Mix and match the ones you need.

Development

git clone https://github.com/273v/kaos-nlp-transformers
cd kaos-nlp-transformers
uv sync --group dev --extra clustering

Install pre-commit hooks (recommended — they run the same checks as CI on every commit, scoped to staged files):

uvx pre-commit install
uvx pre-commit run --all-files     # one-time full sweep

Manual QA commands (the same set CI runs):

uv run ruff format --check kaos_nlp_transformers tests
uv run ruff check kaos_nlp_transformers tests
uv run ty check kaos_nlp_transformers tests
uv run pytest tests/unit -q

Build from source

uv build
uv pip install dist/*.whl

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md for setup, quality gates, pull request expectations, and engineering standards. By contributing you agree to follow the project conduct expectations and certify the Developer Certificate of Origin v1.1 — sign every commit with git commit -s. Please open an issue before starting on a non-trivial change so we can align on scope.

Security

For security issues, please do not file a public issue. Report privately via GitHub Private Vulnerability Reporting or email security@273ventures.com. See SECURITY.md for the full disclosure policy.

License

Apache License 2.0 — see LICENSE and NOTICE.

Copyright 2026 273 Ventures LLC. Built for kelvin.legal.