transformers-v9

V9 Palace Transformer — an independent from-scratch LLM architecture with block-diagonal palace-collated attention, per-palace LoRA K/V adapters, and cross-palace semantic gating. Not a base-model adapter — its own embedding/position/lm_head stack.

Part of the V∞ (SiliconLifeOS) paradigm: cognitive structure is architecture, not prompt engineering.

Architecture

Input → Embed/Pos → 6×PalaceAttentionLayer (residual) → LM Head → Output

Quick Start

import transformers_v9  # registers v9_palace with HF Auto Classes

from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("v9_palace", vocab_size=32768, hidden_size=2048)
model = AutoModelForCausalLM.from_config(config)

Training

from transformers_v9 import V9PalaceTrainer

trainer = V9PalaceTrainer(model, train_dataset, eval_dataset)
trainer.train()

From-scratch training pipeline included (from transformers_v9.train import get_curriculum_dataset):

• SPC L1-L6 curriculum data
• CP fusion tasks
• Multi-domain reasoning

Router (Semantic Palace Routing)

from transformers_v9 import V9RouterConfig, V9Router

router = V9Router(V9RouterConfig())
palace = router.route("Discuss the attention mechanism in Transformers")
# → Palace ID for attention-related reasoning

4-level cascade: L0 domain hash → L1 keyword voting → L2 group Jaccard → L3 embedding (optional).

Install

pip install transformers-v9

Requirements

• Python 3.8+
• PyTorch 2.0+
• transformers 4.30+
• accelerate 0.20+

Status

Alpha prototype — structured reasoning validation complete. CPU training verified (500 samples, 20 epochs: ont_self 0.55→1.11, accuracy 96.5%). GPU training for full-scale (1.5B+) recommended.

Links

• V∞ Paradigm
• GitHub