r3alai

R3AL.AI is the brand. r3alai is the Python package (pip install r3alai); import it as r3alai.

Universal vision-model quantization, architecture-agnostic. Full round trip:

your_model (.pt/.h5/.tflite/...)  →  export to ONNX  →  quantize (PTQ) or qat_pipeline  →  quantized .onnx
                                                                                          ↳  export_native=True → back to a native .pt (torch.nn.Module, any architecture)

No vendor lock-in: export from PyTorch, TensorFlow/Keras, Ultralytics YOLO, TFLite, Paddle, or a ready-made ONNX file. After quantization the result can be rebuilt into a plain .pt file, regardless of the source architecture (ResNet, EfficientNet, ViT, YOLO, arbitrary HuggingFace vision models, ...).

Documentation: docs.r3al.ai

Install

pip install "r3alai[vision]"        # PTQ + QAT on ONNX models
pip install "r3alai[vision,yolo]"   # + Ultralytics YOLO support
pip install "r3alai[all]"           # all export adapters (TF, Paddle, TFLite)

For local development: pip install -e ".[vision]". QAT requires onnx2torch (included in [vision]).

Quick start

from r3alai.quant import Quantizer, QuantConfig

# PTQ: static INT8 or INT4 with calibration images (the default; covers Conv layers)
Quantizer().quantize(
    "yolov8n.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./out",
)

# PTQ: tune how the activation clipping threshold is calibrated
Quantizer(QuantConfig(calibration_method="percentile", calibration_percentile=99.99)).quantize(
    "yolov8n.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./out",
)

# PTQ: no calibration data (MatMul/Gemm only, so a poor fit for Conv-heavy models)
Quantizer(QuantConfig(method="ptq_dynamic")).quantize(
    "efficientnet_b0.onnx", output_dir="./out"
)

# QAT: ONNX + training images (generic pipeline)
Quantizer(QuantConfig(mode="qat")).train_qat(
    "yolov8n.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./qat_out",
    epochs=3,
    export_native=True,
    ptq_runtime=True,  # also produce a genuinely smaller ptq deliverable
)

# QAT: native Ultralytics .pt in (keeps the YOLO detect/pose head intact)
Quantizer(QuantConfig(mode="qat")).train_qat(
    "yolo11n-pose.pt",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./qat_out",
    source="ultralytics",
    epochs=3,
    export_native=True,  # uses YOLO.save() -> loadable with YOLO(path)
)

Two paradigms

PTQ methods (action: quantize)

Calibration methods (ptq_static)

Calibration decides where the INT8 or INT4 range stops and clipping begins. calibration_method picks how that threshold is chosen:

config = QuantConfig(
    method="ptq_static",
    calibration_method="percentile",
    calibration_percentile=99.99,
)
Quantizer(config).quantize("model.onnx", calibration_data=images, max_calib_samples=256)

The histogram methods want 256-512 diverse calibration samples (max_calib_samples caps how many are used, default 100); the SDK warns below that. QAT has the same three options through qat_calib_method. The chosen method and its parameters land in the deliverable's manifest.

ptq_static uses per_channel=True by default and automatically excludes ops close to the graph outputs from quantization (e.g. the Sigmoid/Concat of a detection head). With few calibration images those sensitive tail ops can otherwise collapse accuracy entirely (mAP ≈ 0). Override with nodes_to_exclude or per_channel=False if needed.

Both ptq_static and ptq_dynamic also exclude Softmax/LayerNormalization/Gelu/Erf (plus the MatMul/Gemm feeding directly into or out of a Softmax) from quantization by default: exclude_attention_sensitive_ops=True. Attention softmax and LayerNorm activations have very peaked/small-variance value ranges that saturate under a linear INT8 or INT4 scale; on transformer-style backbones (CLIP ViT, CLIPSeg decoders and the like) this exclusion is the difference between a working model and a full collapse. Set exclude_attention_sensitive_ops=False to disable, or pass an explicit nodes_to_exclude to override both auto-detections.

QAT (action: qat_pipeline)

Universal ONNX in → train with fake-quant layers → ONNX out. No separate method: use wbit, abit, epochs.

from r3alai.quant import Quantizer, QuantConfig

config = QuantConfig(mode="qat", qat_wbit=8, qat_abit=8)
result = Quantizer(config).train_qat(
    "yolov8n.onnx",
    calibration_data=["img1.jpg"],
    output_dir="./qat_out",
    epochs=1,
)

Export to ONNX (any architecture)

export_to_onnx picks the right adapter automatically based on file extension, or force one with source=:

from r3alai.quant.export import export_to_onnx

export_to_onnx("yolo11n-pose.pt", output_dir="./out", source="ultralytics", imgsz=640)
export_to_onnx("resnet18.pt", output_dir="./out", source="pytorch", input_shape=[1, 3, 224, 224])
export_to_onnx("model.h5", output_dir="./out", source="tensorflow")

Back to native .pt (after PTQ or QAT)

Every PTQ backend and QAT pipeline supports export_native=True: the quantized ONNX graph is rebuilt into a plain torch.nn.Module via onnx2torch and saved as .pt. Architecture-agnostic, so it works for any model the adapters above can export, not just YOLO.

result = Quantizer().quantize("resnet18.onnx", output_dir="./out", export_native=True)
# result.path contains both the .quantized.onnx and a reconstructed .pt model

Best-effort: if the reconstruction fails (e.g. exotic QuantizeLinear/QLinear* ops that onnx2torch doesn't know), the quantization job itself does not fail. The error is reported in the manifest (native_export_error) and the .onnx deliverable remains the primary, always-valid result.

QAT folds trained QuantConv2d layers back into plain fp32 Conv2d for export, so the *_qat.quantized.onnx file is float32 on disk. Pass ptq_runtime=True to additionally run an ONNX Runtime static-INT8 or INT4 pass on top: that yields a separate, genuinely smaller deliverable (path reported as ptq_runtime_output_model in the manifest / API response).

Verify your install

python -c "from r3alai.quant import Quantizer, QuantConfig; print('OK')"
python scripts/verify_sdk.py    # from a source checkout: full export → PTQ → QAT → benchmark round trip

RunPod API

The RunPod serverless handler and deploy tooling live in the separate API-SDK repo, which installs r3alai as a dependency and calls the same Quantizer/QuantConfig via a JSON job payload:

{
  "input": {
    "action": "quantize",
    "model": "/runpod-volume/models/your_model.onnx"
  }
}

See that repo's docs/API.md for the full API reference.

MIT