---

💡 The Core Thesis: Why Signal Processing for Transformers?

Modern Transformers suffer from a well-known pathology: extreme outlier activation spikes (e.g., sporadic latent magnitudes exceeding normal bounds by $5\times$ to $10\times$). In standard attention mechanisms:

$$\text{Attention}(Q, K, V) = \text{Softmax}\left(\frac{Q K^\top}{\sqrt{d_k}}\right) V$$

When an unregularized spike detonates in the Query ($Q$) or Output ($O$) projection:

1. It blinds the Softmax operator into an artificial Attention Sink, triggering hallucinations.
2. It inflates logit entropy, forcing models to spawn thousands of speculative "Chain-of-Thought" tokens to resolve internal hesitation.

Existing solutions like QK-Norm (RMSNorm) act as crude, global volume knobs—they divide the entire vector by its root-mean-square, inadvertently drowning subtle prompt constraints in the noise floor.

Harmonic-LLM introduces the Dynamic Latent Resonance Suppressor (DLRS):

• $100%$ Bitwise Transparent Bypass ($0.00\text{ dB}$ at rest): Healthy tokens and subtle constraints flow through untouched with zero phase shift and zero precision loss.
• Surgical Outlier Attenuation: Only when a localized channel block experiences an abnormal energy burst does the sidechain detector trigger an instant, localized 8-band orthogonal DCT notch filter via a Straight-Through Estimator (STE).
• Asymmetric $Q/O$ Targeting: Memory coordinate systems ($K$ and $V$) remain untouched, while active cognitive routing ($Q$) and language synthesis ($O$) are conditioned onto an optimal gravitational manifold.

---

🔬 Empirical Spectral Verification

We instrumented a 48-layer live diagnostic probe tracking $3,072$ layer-step forward executions ($48\text{ layers} \times 64\text{ generation steps}$) on an unquantized 12B model, measuring both the raw baseline and harmonic-llm in real-time.

Figure 1: Live quad-panel diagnostic comparing Native Unpatched Base vs. Harmonic-LLM active filtering under a deliberately aggressive stress-test setting (threshold=1.85, steepness=5.0) that force-opens the gate so the attenuation becomes measurable in the spectrum.

Key Physical Readouts from Figure 1

1. Physical Dynamic Gain Reduction (Top-Left): When the crest threshold is set to catch outlier excursions, the $Q$-projection power density (green line) decouples from the unpatched baseline (red dashed line) and smoothly attenuates by $\sim 0.06\text{ dB} \text{ to } 0.10\text{ dB}$ across all 8 harmonic orders.
2. Outlier Spike Clamping (Bottom-Left): In the unpatched base model (red), multiple isolated spike bursts erupt past $2.05$ (e.g., execution steps $\sim 1800, 2000, 2400$). Under harmonic-llm (blue), these pathological excursions are strictly bounded within a stable, well-behaved envelope.
3. High-Frequency Damping in Output Synthesis (Top-Right): Upper harmonic orders ($5, 6, 7$) in the $O$-projection are physically regularized, suppressing conversational sibilance, formatting leaks, and markdown syntax artifacts.
4. The $2.2$ Natural Attractor Ceiling (Bottom-Right): The periodic comb structure reveals the 48-layer heartbeat across all generated tokens, confirming that the model's natural crest ceiling hovers at $\approx 2.26$, precisely where our standard threshold ($2.2$) operates.

⚠️ Read this before copying 1.85 / 5.0: the figure above is a diagnostic stress test, not the recommended default. 1.85 / 5.0 opens the gate on a large fraction of blocks purely so that the spectral difference is visible. Production defaults are threshold=2.2, steepness=3.0. See Tuning Guide.

---

⚡ Quickstart

Install harmonic-llm directly from PyPI or source:

pip install harmonic-llm
# Or install in editable mode from source:
git clone https://github.com/aifeifei798/Harmonic-LLM.git
cd Harmonic-LLM
pip install -e .

Zero-Inference Hot-Patching (1 Line of Code)

harmonic-llm is completely model-agnostic and non-invasive. You can inject dynamic resonance suppression into any Hugging Face Transformer (Gemma, LLaMA, Qwen, Mistral, DeepSeek) without retraining:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from harmonic_llm import apply_harmonic_eq, remove_harmonic_eq, harmonic_status

# 1. Load any off-the-shelf model
model_id = "Qwen/Qwen2.5-7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

# 2. Inject Harmonic Dynamic EQ (automatic Q/O targeting across all layers)
#    threshold = crest-factor knee, steepness = soft-knee sharpness (both optional)
apply_harmonic_eq(model, threshold=2.2, steepness=3.0)
print(f"Active Channels Monitored: {harmonic_status()}")  # e.g., 56 or 96 channels

# 3. Generate with active resonance suppression
inputs = tokenizer("Explain the Nyquist-Shannon sampling theorem.", return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

# 4. Remove hooks at any time to return to 100% stock base state
remove_harmonic_eq(model)

Calling apply_harmonic_eq again automatically swaps the parameters — the old hooks are torn down first, so you can A/B different tunings inside one process without reloading weights.

---

🎚️ Tuning Guide: threshold and steepness

Both knobs live on the sidechain detector. They do not change model weights, do not require retraining, and are pure inference-time arguments of apply_harmonic_eq(model, threshold=..., steepness=...).

The whole operator is a soft-knee compressor applied per 8-channel macroblock:

$$\gamma = \frac{\text{Peak}}{\text{RMS} + \epsilon}, \qquad \alpha_{\text{dyn}} = 0.90 \cdot \tanh!\big(\beta \cdot \mathrm{ReLU}(\gamma - \theta)\big), \qquad X_{\text{out}} = (1-\alpha_{\text{dyn}})X + \alpha_{\text{dyn}}X_{\text{rec}}$$

where $\theta$ = threshold, $\beta$ = steepness, and $X_{\text{rec}}$ is the DCT-quantized reconstruction of the same block.

Argument	Symbol	Default	Valid range	In one sentence
threshold	$\theta$	2.2	[1.0, 2.83]	Crest factor at which the gate starts to open.
steepness	$\beta$	3.0	(0, ~10]	How hard the gate slams shut once it opens.
block_size	—	8	4 / 8 / 16	DCT block length; changes the crest range, so retune $\theta$ if you touch it.
targets	—	("q_proj", "o_proj")	any suffix tuple	Which attention projections to hook; leave as-is to keep $K/V$ untouched.

1. threshold=2.2 — where the knee sits

threshold is a unitless crest-factor threshold, evaluated per 8-channel block, not per token and not in percentage units.

For an 8-element block the crest factor is mathematically bounded:

$$\gamma \in \left[1,\ \sqrt{8}\right] \approx \left[1,\ 2.83\right]$$

• $\gamma = 1$: perfectly flat block (all 8 channels equal).
• $\gamma \approx 2.83$: a single channel carries all the energy — the pathological spike shape.

That bound is the key to reading the parameter:

• $\theta \le 1.0$ → almost every block sits above the knee (and even a perfectly flat block is filtered once $\theta < 1$). This is no longer an EQ, it is a global low-pass on the latent space. Expect flattened vocabulary and dull reasoning.
• $\theta \ge 2.83$ → the gate can never open; the hook stays installed but is a $0.00\text{ dB}$ no-op. Useful as an ablation control.
• 2.2 (default) → sits just below the empirically measured natural ceiling of a healthy 12B model ($\approx 2.26$ in Figure 1, bottom-right). Ordinary high-energy directions pass untouched; only genuine outlier bursts cross the knee. This is why the default is nearly free.
• 1.85 (Figure 1 setting) → deliberately aggressive stress test. Blocks at $\gamma \approx 2.0$ already receive $\alpha_{\text{dyn}} \approx 0.57$, i.e. more than half of the block is replaced by its DCT reconstruction. Great for making attenuation measurable; too much for daily chat.

Practical band: 1.85 – 2.5. Below ~1.8 the gate starts catching ordinary attention-routing directions rather than pathologies, and output tends to become blander and less specific. Above ~2.4 the effect becomes rare and subtle — measurable, but seldom decisive.

2. steepness=3.0 — how hard the knee bites

steepness is the $\beta$ multiplier inside the tanh, i.e. the slope of the compression curve past the knee. The gate is bounded by design, so $\alpha_{\text{dyn}} < 0.90$ always: at most $90%$ of a block is replaced by the quantized reconstruction, and $10%$ of the original signal always survives. That preserves gradient flow and keeps the STE well-behaved.

• Slope at the knee is $0.90,\beta$ per unit of crest overshoot: 2.7 at $\beta=3$, 4.5 at $\beta=5$.
• Knee width (overshoot needed to reach ~76% of saturation) is roughly $1/\beta$: 0.33 crest units at $\beta=3$, 0.20 at $\beta=5$.
• Low $\beta$ (1.0 – 2.0) → soft knee, gradual, "musical" compression. Smallest risk of discontinuities; preferred for creative writing and long-form prose.
• 3.0 (default) → balanced. Strong enough to clamp real spikes, soft enough to avoid a binary on/off texture.
• High $\beta$ (5.0 – 8.0) → hard-knee limiter. Snappy, decisive suppression of spike storms; the most aggressive useful setting.
• $\beta \to 0$ → gate disabled ($\alpha \equiv 0$). $\beta < 0$ is invalid: the tanh flips sign and the blend turns into amplification. Keep it positive.

Gate response: $\alpha_{\text{dyn}}$ as a function of crest factor

Crest $\gamma$	2.2 / 3.0 (default)	1.85 / 5.0 (Figure 1)	2.0 / 4.0	2.3 / 2.5 (precision-safe)
1.85	0.000	0.000	0.000	0.000
2.00	0.000	0.572	0.000	0.000
2.20	0.000	0.847	0.598	0.000
2.26	0.160	0.871	0.700	0.000
2.40	0.483	0.893	0.830	0.222
2.60	0.750	0.899	0.885	0.525
2.83	0.859	0.900	0.898	0.714

Read it as "fraction of the block replaced by its DCT reconstruction". The default column is empty until $\gamma \approx 2.2$ and only reaches ~0.16 at the model's natural ceiling — that is the $0.00\text{ dB}$-at-rest claim in numbers.

Recommended presets

Scenario	threshold	steepness	Why
General chat / first deployment (default)	2.2	3.0	Knee just under the natural ceiling; near-transparent on healthy tokens.
Long chain-of-thought, runaway hesitation tokens	1.95 – 2.05	4.0 – 5.0	Catches the mid-size bursts that inflate logit entropy instead of only extreme spikes.
Spike storms / unstable small models	2.0	4.0	Hard-knee limiter; clamps repeated excursions without lowering $\theta$ too far.
Code, math, structured output (minimal drift)	2.30 – 2.45	2.0 – 3.0	Fewest blocks touched; protects exact syntax and numeric literals.
Creative / long-form writing	2.25	1.5 – 2.0	Softest knee, no audible switching texture.
Diagnostic figure / benchmarking (Figure 1)	1.85	5.0	Force-open the gate so gain reduction is visible in the spectrum.
Ablation control (hook installed, never fires)	2.83	any	Establishes the $0.00\text{ dB}$ reference.

Calibration recipe (4 steps)

1. Baseline first. Run your prompt set with remove_harmonic_eq, record outputs/seeds.
2. Start at the default threshold=2.2, steepness=3.0. If the crest histogram you measure never reaches 2.2, the hook is already a no-op for your model — lower $\theta$ before touching $\beta$.
3. Move $\theta$ in steps of 0.05, not 0.3. Crest factor is a ratio, and the useful band is narrow (1.85 – 2.5).
4. Then shape with $\beta$. Raise $\beta$ if spikes still break through; lower it if output feels mechanically smoothed or repetitive.

Rules of thumb:

• Symptom "still hallucinating on the same prompt" → lower $\theta$ by 0.05 first, raise $\beta$ only after.
• Symptom "answers got generic / lost detail" → raise $\theta$ by 0.05 – 0.1, drop $\beta$ to 2.0.
• Symptom "output oscillates between two styles" → lower $\beta$ (softer knee), keep $\theta$ the same.
• If you change block_size, the crest ceiling becomes $\sqrt{n}$ where $n$ is the block length ($2.0$ for 4, $2.83$ for 8, $4.0$ for 16), so rescale $\theta$ accordingly.

Tuning in code

from harmonic_llm import apply_harmonic_eq, remove_harmonic_eq

# Conservative production default (near bit-exact)
apply_harmonic_eq(model, threshold=2.2, steepness=3.0)

# Aggressive: long CoT / spike storms
apply_harmonic_eq(model, threshold=1.95, steepness=4.5)

# Precision-safe: code & math
apply_harmonic_eq(model, threshold=2.35, steepness=2.5)

# A/B control: hook present but mathematically inert
apply_harmonic_eq(model, threshold=2.83, steepness=3.0)

Every call prints the number of patched channels and the active settings, and re-injection replaces the previous configuration in place.

---

🧮 Mathematical Formulation

1. Crest Factor Sidechain Detector

Activations $X \in \mathbb{R}^{B \times S \times D}$ are partitioned along the feature channel dimension into 8-dimensional macroblocks $X_{\text{blk}} \in \mathbb{R}^{\cdots \times N_{\text{blk}} \times 8}$:

$$\text{Peak} = \max(|X_{\text{blk}}|), \quad \text{RMS} = \sqrt{\frac{1}{8} \sum_{i=1}^{8} x_i^2 + \epsilon}$$

$$\gamma = \frac{\text{Peak}}{\text{RMS} + \epsilon} \quad (\text{Crest Factor})$$

Because RMS is bounded by Peak, $\gamma \in [1, \sqrt{8}]$ for 8-channel blocks — the domain that gives threshold its meaning.

2. Differentiable Soft Dynamic Gate

When $\gamma \le \theta$ (threshold, default $2.2$), dynamic reduction is zero. When $\gamma > \theta$, the gate activates smoothly:

$$\Delta\gamma = \mathrm{ReLU}(\gamma - \theta)$$

$$\alpha_{\text{dyn}} = 0.90 \cdot \tanh(\beta \cdot \Delta\gamma) \quad (\beta = \text{steepness},\ \text{default } 3.0)$$

• $\beta$ sets the knee slope ($0.90\beta$ at $\gamma = \theta$) and the knee width ($\approx 1/\beta$).
• $\alpha_{\text{dyn}}$ is capped at $0.90$, so $10%$ of the original block is always retained.

3. Orthonormal 1D-DCT Type-II Transform Coding

The macroblock is projected onto the discrete cosine frequency basis $M \in \mathbb{R}^{8 \times 8}$:

$$M[j, n] = c_j \cos\left(\frac{\pi}{8} \left(n + \frac{1}{2}\right) j\right), \quad c_j = \begin{cases} \sqrt{1/8} & j = 0 \ \sqrt{2/8} & j > 0 \end{cases}$$

$$C = X_{\text{blk}} M^\top, \quad C_q = \mathrm{STEQuant}(C), \quad X_{\text{rec}} = C_q M$$

4. Broadcasted Channel Blending

$$X_{\text{out}} = (1 - \alpha_{\text{dyn}}) \cdot X_{\text{blk}} + \alpha_{\text{dyn}} \cdot X_{\text{rec}}$$

• Quiescent State ($\alpha_{\text{dyn}} = 0$): $X_{\text{out}} \equiv X_{\text{blk}}$ (exact $0.00\text{ dB}$ bypass).
• Burst State ($\alpha_{\text{dyn}} > 0$): outlier harmonics are clamped and redistributed into a slowly-varying subspace.

---

🥊 Zero-Shot Empirical Benchmarks

Trained with Ghost Harmonic Guidance on an RTX 5090 D and merged statically (merge_and_unload) into GGUF Q8_0, a 12B model equipped with Harmonic Q/O shaping exhibits unprecedented zero-shot cognitive performance:

Benchmark Challenge	Stock Base 12B	Harmonic-LLM 12B	Result Breakdown
The Sally Sisters Logic Trap (Negative deduction)	❌ Failed (4 Sisters) Collapses into local semantic counting hallucination.	🌟 100% Passed (2 Sisters) Captures set-equivalence: "the sisters are the same people".	Zero-Shot Precision
Nonlinear DSP Aliasing Math (14 kHz tanh at 44.1 kHz)	⚠️ Passed with 1,625 CoT tokens (20 s delay) Repeated scratchpad checks, unit typo (Hz vs kHz).	🌟 Instant Direct Pass (400 tokens / 4 s) Zero CoT needed. Exactly derives 3rd fold ($2.1\text{ kHz}$) and 5th double-wrap ($18.2\text{ kHz}$).	$4\times$ Faster, $75%$ Token Savings
Distributed WAL Recovery (Linux Page Cache vs NVMe Flush)	Generic high-level summary with format drift.	🌟 Staff-Level Verdict Identifies flusher threads, NIC TX rings, and lack of global I/O-network priority order.	Surgical Systems Insight
Negative Formatting Constraints (No Markdown backticks allowed)	❌ Failed Stubbornly wraps output in fenced Python blocks.	🌟 100% Followed Pure executable text output without conversational fluff.	Zero Markdown Leaks

---

📂 Repository Layout

Harmonic-LLM/
├── harmonic_llm/
│   ├── __init__.py        # Package exports (apply_harmonic_eq, remove_harmonic_eq)
│   ├── core.py            # Orthonormal DCT-II, crest-factor gate, STE quantizer
│   └── patch.py           # Universal Q/O attention forward-hook injector
├── doc/
│   └── harmonic_package_verification.png   # 4-panel spectral validation plot
├── LICENSE
├── pyproject.toml         # Modern packaging configuration
└── README.md

---

📜 Lineage & Origins

Harmonic-LLM emerged from an unexpected discovery during the development of flashcodec-kv, a research project exploring image-compression codecs for KV Cache storage. While storage-level KV quantization reached diminishing returns against native row-wise integer formats, an internal mechanism study revealed that orthogonal transforms redistribute activation error into a slowly-varying subspace that Softmax barely penalizes.

By pivoting the transform from passive KV storage into active $Q/O$ representation wave-shaping, Harmonic-LLM turned a storage negative result into a major capability breakthrough in reasoning and hallucination suppression.

---

🤝 Citation

If you find this signal-processing approach to Transformer latent manifolds useful in your research or production infrastructure, please cite:

@software{feifei2026harmonicllm,
  author = {Feifei},
  title = {Harmonic-LLM: Acoustic Dynamic Resonance Suppression & Wave-Shaping for Transformer Latent Manifolds},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/aifeifei798/Harmonic-LLM}
}