hypernix

End-to-end toolkit for training ai models on modern or old devices, originaly for converting hypernix.1 into gguf, now for all around training

What's fixed in this update

See changlog.md

Table of contents

• What's fixed in this update
• Module reference
• What's new in v0.70.5
• What's new in v0.70.4
• Install
• Quickstart
• Python API tour
• CLI reference
• Supported model families
• Examples
• Wiki / deep dives
• How the GGUF pipeline works
• Platform notes
• Build / release
• Usage & Documentation
• License

Cross-platform: Linux, macOS, Windows. Python 3.10 - 3.14.

Module reference

Click a category below to expand it.

What's new in v0.70.5

Eleven major additions:

• hnx / hypenix Wiki CLI — Auto-generating documentation browser. hnx shows all modules; hnx <module> shows docs; hnx -q <module> streams quick mode; hnx -b opens in browser. Docs regenerate from source docstrings, so they can't drift out of sync with the code.
• hnx vera — Module verification: syntax check, docstring coverage, type annotations, smoke test. hnx vera <file> or hnx vera --all.
• pressure_cooker_v5 — ORCP optimizer family with int8-quantized momentum (~75% smaller than fp32, ~87% smaller total optimizer state than AdamW -- see the efficiency paper), QAT (Q4/Q5/Q6/Q8), Multi-Token Prediction, EMA shadowing, and the ground-up 3D-ORCP PressureCookerV5S variant (v0.70.6).
• mtp — Multi-Token Prediction for 1.5-3x training efficiency. Sequential/independent modes, shared/independent heads, native workshop integration.
• scavenger — HuggingFace dataset discovery with keyword search, storage budgets, quality filtering (likes/downloads/age), and relevance scoring.
• Freezer QAT support — suggest_qat_batch_size(), prepare_for_qat(), per-bit-width VRAM multiplier profiles.
• Workshop native MTP — attach_mtp_head() and compute_mtp_loss() built into WorkshopFramework.
• tvtop++ fixes — Eliminated border flicker (layout built once), added _block_history_bar re-export, implemented small_mode, fixed self-process filtering.
• New wiki pages — Pressure-Cooker-V5, MTP, Scavenger
• Kitchen.md updated — Added scavenger, MTP, and QAT sections
• Training benefits chart — See below

Training Benefits vs Complexity

Key insight: MTP + Speculative Decoding offer the highest benefit-to-cost ratio. Int8-quantized momentum cuts the momentum buffer's own memory by 75% versus fp32, and PressureCookerV5/V5S's factored curvature keeps the rest of the optimizer state small too -- measured optimizer-state memory lands around 12-13% of AdamW's (see the efficiency paper for the exact numbers and methodology). The trade-offs -- including step-time overhead on some hardware -- are real and are covered in the paper rather than summarized as a single percentage here.

What's new in v0.70.4

Seven additions in the 0.70.4 series:

• qa — QAProcessor formats Q&A datasets into causal LM training strings with optional salt/pepper seasoning
• stml — Short Term Memory Loss: STML context manager (segment folding, untrained hard cap) + calculate_vram_context VRAM calculator with CLI
• TurboAbbicus — exponential curriculum regulator with configurable hard cap, sine-wave oscillation (CPU-adjusted, never GPU), and VRAM safeguard
• tvtop++ fixes — layout tree bug (border shifting on refresh), colors matching original tvtop (CPU=green, RAM=magenta, GPU=red), dynamic console resizing, dynamic graph/log widths
• hypernix stml CLI subcommand — VRAM context calculator from the shell
• hypernix train run new flags — --use-abbicus, --use-turbo-abbicus, --use-stml, --untrained-max-context, --segment-length
• CodeOven.train() new kwargs — use_turbo_abbicus, use_stml, untrained_max_context, segment_length

Earlier: v0.70.0

Five new modules + major optimizer rewrites:

• abbicus — Automatic token regulation and curriculum tuning for model sizes 0.5B–72B
• compute_framework — Hardware-agnostic multi-device training with auto DDP/ZeRO wrapping (CUDA/MPS/CPU/TPU)
• pressure_cooker V2 — Quantization-aware training with fp16/bf16/fp64 mixed-precision, QAT hooks for Q8/Q6/Q5.5/Q4M, plus 10 upgrades (mixed-precision autodetect, QAT hooks, gradient-checkpointing integration, adaptive per-layer gradient clipping, EMA weight shadowing, DDP/FSDP-aware distributed training, dynamic loss scaling with overflow backoff, parameter freeze/unfreeze callbacks, an LR finder, and metrics streaming to tvtop)
• pressure_cooker_v3 — ZeRO-1/2 optimizations, FP8 support, QuantDtype enum + QuantConfig dataclass
• workshop — Model frameworks for TTS/ASR/LLM/Vision with pre-built templates, nano-nano collection support, 30+ architectures (LiquidAI LFM2.5, MiniCPM5, Gemma 4, Qwen3.5, Phi-4, DeepSeek-V2.x, and others)
• tvtop — Now launches the premium tvtop_plus_plus dashboard by default; use tvtop-old for the classic view

Install

From PyPI:

pip install "hypernix[llama-cpp]"     # + bundled llama-cpp-python
pip install "hypernix[train]"         # + transformers, accelerate
pip install hypernix                  # core only

Need a specific torch build? Install torch first; pip will reuse it rather than replace it:

# CUDA 11.8 — old drivers, Pascal GPUs (GTX 1080 et al.)
pip install --index-url https://download.pytorch.org/whl/cu118 torch
pip install hypernix

# CUDA 12.x — modern default
pip install --index-url https://download.pytorch.org/whl/cu124 torch
pip install hypernix

# CPU-only
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install hypernix

# Old Intel Mac / torch 1.13 — the compat shim takes over.
pip install --index-url https://download.pytorch.org/whl/cpu 'torch==1.13.1'
pip install 'hypernix[legacy-torch]'

The main install_requires is torch>=1.13,<3 — 2.7+ is the recommended version (native nn.RMSNorm, fused SDPA), but 1.13+ works via hypernix.torch_compat. See wiki/macOS-legacy.md for the full story.

Sanity-check the environment:

hypernix doctor          # report
hypernix doctor --fix    # install missing runtime deps

Automatic dependency management can be disabled with HYPERNIX_AUTO_INSTALL=0.

Quickstart

Chat with any supported model

hypernix chat --repo-id nix2.5 --message "hello"
hypernix chat --repo-id qwen3.5-4b --message "explain rotary embeddings"
hypernix chat --repo-id gemma-4-e4b --message "write a haiku"

Short names resolve via KNOWN_MODELS; see Supported model families.

Convert a snapshot to GGUF

# Default: fp32 + fp16
hypernix --repo-id ray0rf1re/hyper-nix.1 --output-dir ./out

# Opt in to k-quants (needs llama-quantize)
hypernix --repo-id ray0rf1re/hyper-nix.1 --output-dir ./out \
    --quants fp32 fp16 q8_0 q6_k q4_k_m

Train HyperNix 1.5 (~92.1 M params) on a GTX 1080

python examples/train_hypernix_1_5_gtx1080.py \
    --dataset corpus.txt \
    --tokenizer-source ./hyper-nix-v1 \
    --out-dir ./hypernix-1.5 \
    --steps 2000 --batch-size 1 --context-length 1024

Auto-detects compute capability 6.x, forces fp16 (Pascal has no native bf16), disables TF32 / SDPA / torch.compile, and wraps the training loop in a FlashFreezer so OOMs pause-and-halve rather than crash. See wiki/Pascal.md for the full Pascal playbook.

Build a HyperNix 0.1.5 evaluator

python examples/train_hypernix_0_1_5_evaluator.py --out-dir ./eval

Synthesizes a judge-training corpus with mediocre_fridge, freezes embeddings with old_fridge, trains via oven.train, reloads with the other oven, plots the loss curve with new_fridge. Self-contained smoke test for every subsystem.

Python API tour

import hypernix
from hypernix import freezer, old_oven, old_fridge, mediocre_fridge, new_fridge

# 1) Auto-pick a VRAM strategy.  On a GTX 1080 this returns OldFreezer(fp16);
#    on a 3090 it returns NewFreezer(fp32 / bf16 on Ampere).
fz = freezer.flash_freezer(base=freezer.auto_freezer(), slow=True)

# 2) Preheat an oven from a short name (downloads on first call).
oven = old_oven.preheat(repo_id="nix2.5", device="cuda", dtype="float16")

# 3) Memory hygiene.
old_fridge.freeze(oven.model, patterns=("embed_tokens",))
print(old_fridge.parameter_stats(oven.model))

# 4) Training data.
dataset = mediocre_fridge.synthesize_judge_corpus(n=1024, out_path="judge.txt")

# 5) Train inside a FlashFreezer so OOMs don't blow up the run.
fz.guard(lambda: oven.train(dataset, "./trained", steps=500, batch_size=1))

# 6) Graph.
import pathlib
log = pathlib.Path("./trained/train.log").read_text()
new_fridge.plot_loss_curve(new_fridge.parse_training_log(log), "loss.png")

CLI reference

hypernix <subcommand> [options]

  all                   download -> convert -> [quantize]   (default)
  download              fetch a HuggingFace snapshot
  convert               produce fp32 / fp16 GGUF from a snapshot
  quantize              run llama-quantize on an fp16 / fp32 GGUF
  verify                read-validate a GGUF and print headers
  info                  package + optional GGUF header summary
  upload                push files to a HuggingFace repo
  doctor                environment diagnostic  (pass --fix to install deps)
  fetch-llama-quantize  pre-seed the llama-quantize cache
  train init            create a fresh HyperNix snapshot
  train expand          warm-start a bigger model from a smaller one
  train run             minimal causal-LM training loop
  generate              sample text from a local snapshot
  oven                  code-generation wrapper (preheat + complete / fill)
  chat                  interactive chat REPL against any supported model
  hyped+ / hyped-pro    Node.js TUI agent CLI w/ real cloud+local model dispatch, /gui desktop mode
  stml                  VRAM trained context length calculator

train run accepts curriculum / context management flags:

hypernix train run --model-dir ./snap --dataset data.txt --out-dir ./out \
    --use-turbo-abbicus \        # exponential curriculum (--use-abbicus for linear)
    --use-stml \                 # fold long sequences into batch segments
    --untrained-max-context 16384 \
    --segment-length 512

Quant aliases accepted by --quants and hypernix quantize (v0.51.3 ships 49 aliases mapping to 30 distinct quant types — the table below shows the headline subset; hypernix.quant_list_types() returns the full list at runtime, and hypernix.QUANT_CATALOG[name] gives you the full QuantSpec for any one):

Supported model families

Short names (CLI & Python)

Pass any of these to hypernix chat --repo-id, old_oven.preheat, download_model, etc.

The full registry lives in hypernix.KNOWN_MODELS.

ARCH_PRESETS (seeds for new_oven)

new_oven(arch="...", ...) spins a fresh, parametric model in the shape of any of these families:

• hypernix, llama, llama3, llama3.1, llama3.2, llama3.3, llama4
• qwen2, qwen2.5, qwen3, qwen3.5, qwen3.6
• gemma, gemma2, gemma3, gemma4
• mistral, phi3, phi4
• glm4, glm5, glm5.1
• deepseek, deepseek-r1, nemotron, gpt-oss / gptoss
• nix, nix2

Presets are seeds for brand-new parametric models. Loading a pretrained checkpoint for any of these families works without a matching preset because non-HyperNix model_type values route through transformers.AutoModelForCausalLM.

Examples

• examples/quickstart.py — 5-line Python API demo.
• examples/custom_arch.py — arbitrary-size HyperNix.
• examples/upload_to_hub.py — publish to the Hub.
• examples/train_hypernix_0_1_5_evaluator.py — tiny evaluator demo wiring ovens + all three fridges.
• examples/train_hypernix_1_5_gtx1080.py — production-shape 92.1 M model trained on an 8 GB Pascal card.

Wiki / deep dives

Topic-focused reference guides live in the wiki/ directory:

• wiki/Home.md — index
• wiki/Ovens.md — old_oven / new_oven reference
• wiki/Fridges.md — old_fridge / mediocre_fridge / new_fridge
• wiki/Ranges.md — new_range / old_range / industrial_range (labeling rubrics)
• wiki/Freezer.md — VRAM manager (OldFreezer / NewFreezer / FlashFreezer)
• wiki/Alarms.md — smoke alarms (Rads / Gas / Modern / Auto) + CPU / GPU preset tables
• wiki/Kitchen.md — pans / microwave / table / sink / instant pot / coffee maker / pressure cooker / qa
• wiki/Abbicus.md — Abbicus (linear) and TurboAbbicus (exponential + sine oscillation)
• wiki/STML.md — Short Term Memory Loss context manager and VRAM calculator
• wiki/Pascal.md — CUDA 6.1 / GTX 1080 playbook
• wiki/Architectures.md — ARCH_PRESETS and KNOWN_MODELS
• wiki/Training.md — scratch training, expansion, and fine-tuning flows
• wiki/CLI.md — full CLI cheat sheet
• wiki/Quantization.md — GGUF conversion + k-quant pipeline
• wiki/Changelog.md — full per-release version history
• wiki/Pressure-Cooker-V5.md — Pressure Cooker V5 / V5+ / V5S with QAT and MTP
• wiki/MTP.md — Multi-Token Prediction guide
• wiki/Scavenger.md — HuggingFace dataset discovery

How the GGUF pipeline works

1. huggingface_hub.snapshot_download pulls weights + tokenizer files.
2. The converter loads the state dict, infers dimensions from tensor shapes (so any HyperNix size works), and maps tensor names onto llama.cpp's canonical GGUF layout when a recognizable pattern matches (Llama, GPT-NeoX, GPT-2, nanoGPT). Unknown names round-trip verbatim.
3. llama-quantize consumes the fp16 GGUF to produce each k-quant.

The CLI emits exactly one fp16 intermediate and reuses it for every k-quant in the plan.

Platform notes

• Linux: full support, every distro tested on: (Ubuntu, Debian, Arch.)
• macOS: Metal for inference, Homebrew for llama-quantize. (untested)
• Windows: native support; doctor accepts Windows; llama-quantize auto-downloads Windows binaries; use scoop / chocolatey for system deps. (untested)
• Pascal (GTX 1080 / 1080 Ti / Titan Xp): install torch from the CUDA 11.8 index first (see above). Use OldFreezer or auto_freezer(); pascal_safe_dtype() picks fp16. hypernix.freezer.pascal_mode_hints() returns a dict of recommended settings (batch size, dtype, TF32/SDPA toggles) for the detected card.

Build / release

pip install build twine
python -m build
twine check --strict dist/*

Release tags (vX.Y.Z) fire .github/workflows/release.yml which publishes to PyPI via Trusted Publishing and attaches the wheel + sdist + an examples-scripts tarball + SHA256SUMS to a GitHub Release.

Usage & Documentation

Comprehensive performance analysis and training guides available in the following PDFs:

# Reproduce the optimizer benchmarks yourself:
python scripts/benchmark_v5.py              # AdamW vs PressureCookerV5, step time + peak mem
python scripts/benchmark_v5s.py             # AdamW vs V5 vs V5S, step time + peak mem
python scripts/measure_optimizer_memory.py  # exact optimizer-state bytes per parameter tensor

License

LLU-0.1