controlmt 0.1.4

Python SDK for the ControlMT v2.3 Kannada↔English translator

controlmt

Python SDK for ControlMT v2.3 — a compact 139M-parameter Kannada ↔ English translator.

Install

# CPU-only (smaller — ~200 MB torch wheel)
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install controlmt

# GPU (CUDA) — default; pulls ~2 GB CUDA torch wheel
pip install controlmt

The default pip install controlmt works on both CPU and GPU machines — but pulls the GPU-enabled torch wheel (~2 GB). If you don't have a CUDA GPU, install the CPU-only torch wheel first (one extra command, ~200 MB) and the model still runs at full speed via bf16 / int8 dynamic quantization.

Quick start

from controlmt import ControlMT

model = ControlMT.from_hf()                                    # auto everything
print(model.translate("ನಾನು ಕನ್ನಡ ಮಾತನಾಡುತ್ತೇನೆ."))
# → "I speak Kannada."

That's it. The SDK auto-detects:

• Device — GPU if CUDA is available, else CPU (overridable)
• Dtype — fp16 on GPU, bf16 on CPU (overridable; bf16 only when supported)
• Direction — Kannada → English if input is mostly Kannada chars, else reverse

Explicit control

# Force CPU, even on a GPU box
model = ControlMT.from_hf(device="cpu")

# Force GPU; falls back to CPU with a warning if not present
model = ControlMT.from_hf(device="gpu")

# Pick an exact dtype
model = ControlMT.from_hf(device="cpu", dtype="bf16")    # bfloat16
model = ControlMT.from_hf(device="gpu", dtype="fp16")    # float16
model = ControlMT.from_hf(dtype="fp32")                  # full precision

# CPU int8 dynamic quantization (~2× faster than bf16 on CPU)
model = ControlMT.from_hf(device="cpu", quant="int8")

# Specific HF revision (e.g. a pre-quantized branch)
model = ControlMT.from_hf(model_id="anandkaman/controlmt-v2.3-int8", quant="int8")

# Loud: print the auto-pick decisions
model = ControlMT.from_hf(verbose=True)

Inspect the resolved config:

>>> model
<ControlMT model_id='anandkaman/controlmt-v2.3' cuda · float16>
>>> model.config
ResolvedConfig(device='cuda', dtype_str='float16', quant='none', bf16_cpu=True)

Batched translation

By design: you must specify batch_size to opt into batching. Otherwise the SDK runs one sentence at a time — predictable memory, no surprises.

texts = ["ನಾನು ಕನ್ನಡ.", "I speak English.", ...]

# Default: one at a time (safe everywhere)
outs = model.batch_translate(texts)

# Explicit fixed batch size
outs = model.batch_translate(texts, batch_size=8)

# Auto-pick batch size from free VRAM (GPU only)
outs = model.batch_translate(texts, auto_batch=True)

Mode	CPU	GPU
(no batch_size, no auto_batch)	1 sentence at a time	1 sentence at a time
batch_size=N	uses N	uses N
auto_batch=True	ignored + warning → 1	probes torch.cuda.mem_get_info(), picks N ≤ 64

Other endpoints

# Heuristic direction detection (>30% KN chars → kn2en, else en2kn)
ControlMT.detect_direction("ನಾನು ಕನ್ನಡ.")    # → "kn2en"

# JIT/compile warmup — kills the 5–10s "first request" lag in production
model.warmup()

# Run the 6-pair DEPLOYMENT.md benchmark suite on YOUR hardware
result = model.benchmark()
# {'config': 'cuda · float16', 'num_beams': 2, 'median_latency_s': 0.19, 'rows': [...]}

Architecture note

ControlMT v2.3 is an encoder-decoder seq2seq model (T5/mBART family), not a decoder-only LM. That means:

• ✅ Works: this SDK, raw Transformers, FastAPI, Docker, HF Inference Endpoints
• ❌ Doesn't work without significant adapter work: vLLM, Ollama, llama.cpp/GGUF, HF TGI

See DEPLOYMENT.md Section 9 for the full "not supported" table and why.

License

Apache 2.0. Same as the underlying model weights.