suave-torch 0.1.0

SUAVE: Sparse Uncertainty-Aware Vector Encoding — A novel LLM algorithm that solves hallucination at the architectural level

suave-torch

SUAVE — Sparse Uncertainty-Aware Vector Encoding

A novel LLM algorithm that solves hallucination at the architectural level.

The Problem

Every transformer-based LLM shares one fundamental flaw: the softmax attention function forces the model to always blend token representations — even when no relevant context exists. This is the mathematical root cause of hallucination. Additionally, LLMs internally generate uncertainty signals but these are silently discarded and never reach the output.

The SUAVE Solution

SUAVE combines two architectural innovations:

1. Sparse Selective Attention (replaces softmax) Uses entropy of the attention distribution to detect uncertainty. High entropy = uncertain = abstain signal raised. Low entropy = confident = normal attention proceeds.

2. Uncertainty Channel (new parallel pathway) A dedicated channel that carries uncertainty signals from the attention layer all the way through to the output logits. When internal uncertainty is high, output is automatically hedged.

Key Results

Metric	Value
Separation Gap	0.7113
Confident sequence uncertainty	0.1281
Uncertain sequence uncertainty	0.8394
Uncertainty loss (epoch 1)	0.1864
Uncertainty loss (epoch 30)	0.0216

Installation

pip install suave-torch

Quick Start

from suave import SUAVEConfig, SUAVELanguageModel, SUAVETrainer
from suave.calibration import UncertaintyVisualizer

# Configure
config = SUAVEConfig(
    vocab_size=1000,
    embed_dim=128,
    num_heads=4,
    num_layers=3,
    ff_dim=256,
    max_seq_len=64
)

# Build
model = SUAVELanguageModel(config)
model.count_parameters()

# Train
trainer = SUAVETrainer(model, config)
history = trainer.fit(train_data, train_labels)

# Evaluate
results = trainer.evaluate(test_data, test_labels)

# Visualize
UncertaintyVisualizer.plot_distribution(
    confident_scores, uncertain_scores
)
UncertaintyVisualizer.plot_training_curves(history)

Architecture

Input Tokens ↓ Token + Position Embeddings ↓ ┌─── SUAVE Block × N ────────────────────┐ │ SparseSelectiveAttention │ │ (entropy-based uncertainty gating) │ │ ↓ ↓ │ │ Attention Uncertainty Signal │ │ Output ↓ │ │ ↓ UncertaintyChannel │ │ └──────────↓ │ │ Modulated Hidden States │ │ ↓ │ │ FeedForward + Residual │ └────────────────────────────────────────┘ ↓ LayerNorm ↓ Output Head (logits modulated by uncertainty) ↓ (logits, uncertainty_scores)

Author

Shabbir — Computer Engineering, Government Polytechnic Dahod, Gujarat Algorithm designed and proven: May 2026

License

MIT License