Fusion Check — The Fusion Threshold Law (FAD)

Author: Jose Miguel Madueño Ortega · MIT · free and open

Fuse trained models with a predictable criterion. Before fusing, the Fusion Threshold Law decides whether your pair of models is compatible, which mix (α) to use, and the expected per-task retention — without running the models, without evaluation data, in under one second. If the law approves, it merges for you (native weight fusion or mergekit) and leaves the model ready as GGUF for llama.cpp.

fusion-check detectar                                    # what can my machine run?
fusion-check verificar base/ m1/ m2/ --acc1 0.76 --acc2 0.83   # the law decides
fusion-check fusionar  base/ m1/ m2/ --acc1 0.76 --acc2 0.83   # native merge + GGUF
fusion-check probar fusion_xxx/model "Once upon a time"         # generate text

The scientific paper is in paper/ — The Fusion Threshold Law (CC BY 4.0), with formal (z3) and empirical validation.

The law

Let τᵢ be the task vector of model i and the fusion τ_f = α·τ₁ + (1−α)·τ₂. In the high-dimensional limit, the retention of task i after fusing obeys:

R_i  ≈  1 − (1−α)² · γᵢ        with   γᵢ = 2(1 − cos²θ_ij) / d_eff
E[e²] = 2(1−α)²·(1−cos²θ)/d_eff        (mean task error under convex fusion)

• θ_ij — angle between task vectors (real cosine of the weight deltas)
• d_eff — effective dimensionality of the task kernel
• Threshold: cos²θ ≥ 1 − δ²·d_eff/(2(1−α)²) separates "preserves" from "degrades"

With per-side calibrated retention (13 measured pairs, LOOCV corr 0.81, 9/13 correct decisions):

R̂ᵢ = 0.8035 + 8.3267·c − 0.0229·acc_i − 0.2096·acc_j

and the closed-form optimal mixing coefficient (maximizes min(R₁,R₂), verified symbolically and numerically):

α* = √γ₁ / (√γ₁ + √γ₂)

The law is established by three independent routes: analytic derivation, formal verification with the z3 SMT solver (NLSat, real nonlinear arithmetic — theorems T1–T3 plus 28 supporting theorems on projector composition), and empirical validation on real models (embedding-space check on a live transformer with machine-precision agreement ~1e−16; 13-pair fine-tuning study).

Decision policy (FAD v2)

Install

git clone <this-repository>
cd fusion-check
pip install -e .                # core (verify, detect, catalog)
pip install -e ".[gguf]"       # + GGUF conversion (gguf, sentencepiece)
# mergekit is optional (pip install mergekit): the tool falls back to native
# weight fusion, which needs no extra dependencies.
# llama.cpp (https://github.com/ggml-org/llama.cpp) is needed only to convert
# to GGUF (its convert_hf_to_gguf.py script) and to probe GGUF models.

Usage

# 1) What can my machine run? (RAM, CPU, disk → models that fit)
fusion-check detectar

# 2) The law decides — no merging, no evaluation data, < 1 s
fusion-check verificar tiny-stories-base/ ft_a/ ft_b/ --acc1 0.76 --acc2 0.83

# 3) If the law approves: merge → GGUF ready for llama.cpp
fusion-check fusionar tiny-stories-base/ ft_a/ ft_b/ --acc1 0.76 --acc2 0.83

# 4) Probe the fused model (transformers, or llama-cli for .gguf)
fusion-check probar fusion_ft_a__ft_b__a0.50/model "Once upon a time"

Pass --d <d_eff> if you know the effective dimensionality of the task kernel; otherwise it is estimated from the delta spectrum (the full calibrated decision needs the fine-tune accuracies).

Why it exists

Blind "soup" (weight averaging) always merges — and when the models are specialists of different tasks, it degrades. FAD provides the necessary and sufficient condition on the task kernel to know beforehand whether the fusion preserves behavior, and the α* to balance it. Demonstrated on a real model (TinyStories-3M, LoRA): two weak, diverse estimators of the same task (0.756/0.762) → soup 0.822; two orthogonal specialists → the law says NO_MERGE and prevents the dilution.

Honesty

• The per-side predictor is calibrated on 13 pairs (LOOCV 0.81): reliable outside the gray zone; inside it, it tells you (confidence: media/baja).
• Fusing specialists of different tasks does not synthesize capability: it dilutes. The real gain is soup of estimators of the same task.
• Out-of-calibration cosines fall back to the pure law with a low-confidence flag; the estimated d_eff is labeled as optimistic.
• Nothing is uploaded or run on servers: everything is local.

Verification

python -m pytest tests/          # 13 lab pairs + policy + α* + brand

License

MIT — see LICENSE. No patents: the method and the code are free (decision of the author, 2026).

Fusion Threshold Law — discovered by Jose Miguel Madueño Ortega (2026).