dno 0.5.3

Dynamic Neural Organism (DNO): A self-evolving, growing, and pruning neural network framework.

DNO: Dynamic Neural Organism 🧬

"Don't just train a model. Raise an organism."

DNO (Dynamic Neural Organism) is a PyTorch framework that treats neural networks as living, evolving biological entities. Unlike static architectures that are fixed at initialization, a DNO grows new layers, specializes into expert lobes, and prunes dead tissue — all in real-time, based on the complexity of the data it consumes.

---

📚 Table of Contents

1. Quickstart
2. Core Philosophy
3. Architecture Overview
4. The DNO Lifecycle
   • Phase 1: DNA Configuration
   • Phase 2: Infancy (Scratch Training)
   • Phase 3: Adulthood (Adaptive Growth)
   • Phase 4: Serialization (Save/Load)
5. Advanced Mechanics
   • Surgical Training Control
   • Robust Persistence (Zombie Repair)
   • Dynamic Gradient Locking
   • Auto-Casting
   • Survival Engine (Pruning)
6. API Reference
7. Changelog
8. Contributing & License

---

⚡ Quickstart

pip install dno

import torch
import torch.nn as nn
from dno import (
    OrganismManager, DynamicNetwork, BaseEvolvableModule,
    DnoConfig, GrowthEngine
)

# 1. Configure the organism's DNA
config = DnoConfig(training_phase='adaptive', d_model=64, entropy_threshold=0.5)

# 2. Build the body
manager = OrganismManager()
network = DynamicNetwork(manager, config)

# 3. Plant the seed (any PyTorch module)
seed = BaseEvolvableModule(nn.Linear(64, 64))
seed.dynamic_id = "seed_cortex"
seed.set_specialty("general")
network.add_layer(seed)

# 4. When the organism gets confused, it grows!
growth = GrowthEngine(network, config)
optimizer = torch.optim.Adam(network.parameters(), lr=1e-3)

# Force-grow a new expert lobe
growth.inject_layer("seed_cortex", "expert_math", optimizer=optimizer)

# Focus training only on the new expert
growth.freeze_all_except("expert_math")

print(f"🧬 Organism now has {len(manager.registry)} active layers")
# >>> 🧬 Organism now has 2 active layers

---

🧠 Core Philosophy

Traditional AI is like a Statue: You carve it (define architecture), polish it (train), and it stays that way forever.

DNO is like a Tree: You plant a seed (Seed Cortex). If the environment is rich (complex data), it grows branches (Expert Lobes). If a branch is useless, it withers and gets pruned.

Key Capabilities

Feature	Description
Neurogenesis (Mitosis)	The network physically adds new layers when "confused" (High Entropy)
Specialized Organisms	The brain divides into a "General Core" and task-specific "Expert Lobes"
Dynamic Gradient Locking	Automatically freezes Core during Expert training and vice versa
Surgical Training Control	Manually inject layers, freeze/unfreeze specific experts
Smart Routing	Routes "Familiar" data to the Core and "Novel" data to Experts
Expert Affinity Routing	Tests each expert to find the best match for new data before creating new experts
Adaptive Entropy Growth	Dynamic baseline + spike detection instead of fixed entropy threshold
Robust Persistence	Save/Load with automatic repair of broken or missing modules
Auto-Casting	Safely handles raw Token IDs (LongTensor) by auto-casting to Float32
Survival Engine	Monitors layer utility via Information Gain; prunes dead tissue

---

🏗 Architecture Overview

                    ┌─────────────────────────────────────────┐
                    │         DynamicNetwork (Body)           │
                    │                                         │
  Input ──────────► │  ┌──────────────┐                       │
                    │  │ Seed Cortex  │──── General Core      │
                    │  │  (general)   │                       │
                    │  └──────┬───────┘                       │
                    │         │ Mitosis (Clone + Noise)       │
                    │    ┌────┴────┐                          │
                    │    ▼         ▼                          │
                    │  ┌──────┐ ┌──────┐                     │
                    │  │Expert│ │Expert│  ◄── Specialized     │
                    │  │ Math │ │ Code │      Lobes           │
                    │  └──┬───┘ └──┬───┘                     │
                    │     └────┬───┘                          │
                    │          ▼                              │ ──────► Output
                    │   Smart Aggregation                    │
                    └─────────────────────────────────────────┘

  Managed by:
  ┌───────────────────┐  ┌──────────────┐  ┌─────────────────┐
  │ OrganismManager   │  │ GrowthEngine │  │ SurvivalEngine  │
  │ (Brain Map/Graph) │  │ (Mitosis)    │  │ (Pruning/GC)    │
  └───────────────────┘  └──────────────┘  └─────────────────┘

What happens during Mitosis?

1. Cloning: The parent layer is deep-copied.
2. Mutation: Variance-based noise (from optimizer state) is added to break symmetry.
3. Gamma Gating: The clone enters with gamma=0.0 (IdentityWrapper) so it initially does nothing, then gradually participates.
4. Rewiring: The clone is connected to the same inputs/outputs as the parent (parallel branch).
5. Optimizer Sync: The clone's parameters are registered to the optimizer with a high initial LR that decays (Cold Start).

---

🧬 The DNO Lifecycle

Raising a DNO involves distinct biological phases.

Phase 1: DNA Configuration (DnoConfig)

Before birth, define the organism's genetic constraints.

from dno import DnoConfig

config = DnoConfig(
    # --- Lifecycle ---
    training_phase='scratch',       # 'scratch' (Infancy) or 'adaptive' (Adulthood)
    
    # --- Growth Triggers ---
    entropy_threshold=0.6,          # Confusion > 0.6 → consider growing
    evolution_cooldown_steps=100,   # Min steps between mitosis events
    min_loss_improvement=0.001,     # Required loss drop before allowing new growth
    
    # --- Physical Constraints ---
    max_param_count=100_000_000,    # Cap at 100M parameters
    vram_limit_gb=4.0,             # Stop growing if VRAM exceeds 4GB
    
    # --- Architecture Defaults ---
    d_model=128,                    # Hidden dimension size
    n_heads=4,                      # Attention heads for Transformer blocks
    dropout_rate=0.1,               # Dropout rate
    
    # --- Survival ---
    utility_threshold=0.01,         # Layers below this are marked "dying"
    grace_period=100,               # Steps before dying layers get pruned
)

Tip: Set manual_mode=True to skip auto-scaling of d_model, n_heads, and learning_rate.

Phase 2: Infancy (scratch Training)

Goal: Build a strong generalist core ("Seed Cortex").
Behavior: Growth is DISABLED. The model behaves like a standard, static PyTorch model.

from dno import OrganismManager, BaseEvolvableModule, DynamicNetwork
import torch
import torch.nn as nn

# 1. Initialize
manager = OrganismManager()
network = DynamicNetwork(manager, config)

# 2. Add the Seed Cortex (any PyTorch module)
seed_layer = nn.Sequential(
    nn.Linear(128, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)
seed = BaseEvolvableModule(seed_layer)
seed.dynamic_id = "seed_cortex"
seed.set_specialty("general")  # Mark as General Core
network.add_layer(seed)

# 3. Standard PyTorch training loop — NO growth happens
optimizer = torch.optim.Adam(network.parameters(), lr=1e-3)

for epoch in range(100):
    optimizer.zero_grad()
    output = network(input_data)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()

Phase 3: Adulthood (adaptive Growth)

Goal: Adapt to new, complex tasks by growing specialized organs.
Behavior: The model monitors entropy. High confusion triggers Mitosis.

from dno import GrowthEngine

# 1. Switch Phase
config.training_phase = 'adaptive'

# 2. Initialize Growth Engine
growth_engine = GrowthEngine(network, config)

# 3. Training loop with entropy monitoring
for step in range(1000):
    optimizer.zero_grad()
    output = network(inputs)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()
    
    # Calculate entropy manually (or use your own tracking)
    with torch.no_grad():
        probs = torch.softmax(output, dim=-1)
        entropy = -torch.sum(probs * torch.log(probs + 1e-9), dim=-1).mean().item()
    
    # Check growth trigger (pass recent entropy history)
    entropy_history.append(entropy)
    is_triggered, reason = growth_engine.check_growth_trigger(
        entropy_history=entropy_history[-10:],
        current_step=step,
        current_loss=loss.item()
    )
    
    if is_triggered:
        print(f"🌟 EPIPHANY! Reason: {reason}")
        
        # Trigger Mitosis — clone the seed into a new expert
        growth_engine.mitosis(
            parent_uuid="seed_cortex",
            optimizer=optimizer,
            specialty_tag="expert_coding_v1"
        )
    
    # Decay new layer learning rates
    growth_engine.stabilize_optimizer(optimizer)

Phase 4: Fluid Serialization (Save/Load)

Standard torch.save fails on DNOs because the architecture changes dynamically. Use the .dno format.

# Save everything (Topology + Weights + Config + History)
network.save_dno("my_organism.dno")

# Or save just the state_dict for standard PyTorch compatibility
network.save_standard("checkpoint.pt")

Loading requires a module factory — a function that reconstructs your custom blocks by type name:

from dno import DynamicNetwork
from dno.core.blueprints import SeedBlock

def my_factory(type_name):
    """Reconstructs modules by their class name."""
    if type_name == "SeedBlock":
        return SeedBlock(input_dim=128, hidden_dim=256)
    if type_name == "Sequential":
        return nn.Sequential(nn.Linear(128, 128), nn.ReLU(), nn.Linear(128, 10))
    # Return None to trigger automatic GPTModel fallback
    return None

loaded_network = DynamicNetwork.load_dno("my_organism.dno", module_factory=my_factory)

Note: If the factory returns None or raises an exception for a module type, DNO automatically reconstructs it as a GPTModel Transformer block. See Zombie Repair below.

---

🔧 Advanced Mechanics

Surgical Training Control

Use GrowthEngine to manually direct evolution — don't wait for entropy triggers.

from dno import GrowthEngine

growth_engine = GrowthEngine(network, config)

# 1. Inject a new expert lobe (forced mitosis)
# IMPORTANT: Pass the optimizer so its parameters are registered!
growth_engine.inject_layer(
    parent_tag_or_uuid="seed_cortex",
    new_tag="expert_math_v1",
    optimizer=optimizer,
    base_lr=0.001
)

# 2. Freeze everything EXCEPT the new expert
# This prevents interference during focused training
growth_engine.freeze_all_except("expert_math_v1")

# 3. Train on math dataset...
for batch in math_dataloader:
    optimizer.zero_grad()
    output = network(batch)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()

Robust Persistence & Zombie Repair

DNO is resilient to corruption and missing class definitions. If you load a brain but the factory can't reconstruct a specific layer:

1. Automatic Fallback: The broken layer is replaced with a generic GPTModel (Transformer Decoder block).
2. Deep Scan & Repair: After loading, a full scan identifies and replaces any None or broken modules.
3. Topology Preservation: All graph connections (edges, inputs, outputs) remain intact.

# Even with a broken/incomplete factory, loading succeeds
net = DynamicNetwork.load_dno("brain.dno", module_factory=lambda t: None)
# >> [WARNING] Factory failed for SeedBlock...
# >> [REPAIR] Reconstructing as default GPTModel.
# >> [OK] Organism Resurrected.

Dynamic Gradient Locking 🔒

DNO automatically manages requires_grad to prevent Catastrophic Forgetting:

Data Type	General Core	Expert Lobes
Familiar (CPT)	✅ Training	❄️ Frozen
Novel (SFT)	❄️ Frozen	✅ Training
Scratch Phase	✅ Training	N/A (no experts yet)

This happens automatically inside network.forward() based on entropy-based novelty detection.

Auto-Casting 🛡️

If you pass raw LongTensor inputs (Token IDs) to a module expecting floats:

• DNO performs a Deep Scan of all submodules to check for nn.Embedding layers.
• If an Embedding is found → data is preserved as Long (correct behavior).
• If no Embedding → data is auto-cast to Float32 to prevent dtype mismatch errors.

Survival Engine (Pruning) 🧹

The SurvivalEngine acts as the organism's immune system:

from dno import SurvivalEngine

survival = SurvivalEngine(manager, config)

# Inside training loop:
# 1. Monitor utility (Information Gain via KL Divergence)
for uuid, module in manager.registry.items():
    survival.calculate_information_gain(module, input_tensor, output_tensor)

# 2. Decay dying layers' weights
survival.apply_selective_decay(optimizer)

# 3. Garbage collect dead tissue
removed = survival.garbage_collect(network)
print(f"Pruned {removed} dead layers")

Layers with low utility score enter a grace period. If they don't recover, their weights decay to zero and they're removed from the graph.

Expert Affinity Routing 🎯

v0.5.0 — Instead of training ALL experts on every data type, DNO now automatically discovers which expert best understands the incoming data.

  New Data Arrives
       |
       v
  ┌─────────────────────────────────┐
  │  AFFINITY PROBING               │
  │                                 │
  │  Test Expert 1 → Entropy: 2.1  │ ← Best match!
  │  Test Expert 2 → Entropy: 4.8  │
  │  Test Expert 3 → Entropy: 3.5  │
  └────────────────┬────────────────┘
                   |
         ┌─────────┴──────────┐
         │ Best < Threshold?  │
         ├─── YES ────────────┤
         │ Train Expert 1     │
         │ Freeze all others  │
         ├─── NO ─────────────┤
         │ Create NEW Expert  │
         │ via Mitosis        │
         └────────────────────┘

Usage:

from dno import GrowthEngine

growth_engine = GrowthEngine(network, config)

# Collect some probe data (first N batches)
probe_data = [batch for i, batch in enumerate(dataloader) if i < 10]

# Find the best expert for this data
best_expert_id, results = growth_engine.probe_expert_affinity(probe_data)

if best_expert_id:
    # Expert found! Only that expert will train.
    # network._active_expert_id is already set.
    print(f"Training expert: {best_expert_id}")
else:
    # No expert fits — create a new one
    growth_engine.mitosis(
        parent_uuid="seed_cortex",
        optimizer=optimizer,
        specialty_tag="new_domain_expert"
    )

Selective Gradient Locking with Affinity:

Data Type	Core	Active Expert	Other Experts
SFT + Affinity	❄️ Frozen	✅ Training	❄️ Frozen
SFT (no affinity)	❄️ Frozen	✅ Training	✅ Training
CPT	✅ Training	❄️ Frozen	❄️ Frozen

Adaptive Entropy Growth 📈

v0.4.0 — Instead of a fixed entropy threshold, DNO establishes a baseline from the data and only triggers growth on genuine spikes.

config = DnoConfig(
    training_phase='adaptive',
    entropy_baseline_window=50,   # Steps to establish baseline
    entropy_spike_factor=1.3,     # Growth if entropy > baseline × 1.3
    evolution_cooldown_steps=200,  # Min steps between growths
)

  Step 1-50:    WARMUP → Entropy baseline = 3.0
  Step 51-199:  Monitor → Entropy ~3.1 → No spike → No growth
  Step 200:     Entropy 4.2 > 3.9 (3.0×1.3) → SPIKE → Mitosis!
  Step 201-250: NEW WARMUP → New baseline for new expert

---

📖 API Reference

Core Classes

Class	Module	Description
DnoConfig	dno.config	Dataclass with all organism parameters (growth, survival, hardware limits)
OrganismManager	dno.core.manager	Central graph manager — tracks nodes, edges, and topology
DynamicNetwork	dno.core.network	The executable body — runs forward pass through the organism graph
BaseEvolvableModule	dno.core.base	Wrapper that adds biological metadata and energy tracking to any nn.Module
GrowthEngine	dno.core.growth	Handles mitosis, entropy triggers, optimizer sync, and surgical controls
SurvivalEngine	dno.core.survival	Monitors utility, applies weight decay, garbage collects dead layers

Blueprint Classes

Class	Module	Description
SeedBlock	dno.core.blueprints	Specialized block with genetic masking (partial layer freezing)
IdentityWrapper	dno.core.blueprints	Gamma-gated residual wrapper — new layers enter as identity ops (gamma=0)
GPTModel	dno.core.blueprints	Standard Transformer Decoder block — used as default fallback during load

Key Methods

Method	Class	Description
add_layer(module, parents)	DynamicNetwork	Adds a layer to the organism graph
save_dno(path)	DynamicNetwork	Saves topology + weights + config as .dno zip
save_standard(path)	DynamicNetwork	Saves only the state_dict (.pt format)
load_dno(path, factory)	DynamicNetwork	Class method — loads .dno with auto-repair
mitosis(parent, optimizer, ...)	GrowthEngine	Clones a layer with smart noise and rewiring
inject_layer(parent, tag, optimizer)	GrowthEngine	Manual mitosis trigger
freeze_all_except(tag)	GrowthEngine	Freezes all modules except those matching tag
check_growth_trigger(entropy, step)	GrowthEngine	Returns (bool, reason) for growth decision
probe_expert_affinity(data_batches)	GrowthEngine	Tests each expert, returns (best_id, results)
set_active_expert(expert_id)	DynamicNetwork	Activate only this expert (+ core)
clear_active_expert()	DynamicNetwork	Reactivate all experts
get_module(tag_or_uuid)	OrganismManager	Lookup by specialty tag or UUID
list_organs()	OrganismManager	Returns summary of all active layers
garbage_collect(network)	SurvivalEngine	Removes dead layers from graph

Utility Functions

Function	Module	Description
load_dno(path, factory)	dno	Top-level shortcut for DynamicNetwork.load_dno
print_organism_status(manager)	dno.utils.dashboard	Pretty-prints organism anatomy to terminal

---

📋 Changelog

v0.5.0 — Expert Affinity Routing

• New: probe_expert_affinity() — tests each expert to find the best match for new data
• New: set_active_expert() / clear_active_expert() — selective expert activation
• New: Selective gradient locking — only the matched expert trains
• New: Forward pass filtering — non-active experts skipped for isolated probing
• New: affinity_probe_steps config parameter

v0.4.x — Adaptive Entropy Growth

• New: Baseline + spike detection replaces fixed entropy threshold
• New: entropy_baseline_window, entropy_spike_factor config parameters
• New: Post-growth baseline reset for new expert warmup
• Fixed: Cooldown now properly resets after each mitosis event
• Fixed: Explicit _growth_locked_until lock prevents rapid sequential mitosis
• New: Debug logging for growth trigger blocking reasons

v0.3.x — Bug Fixes & Stability

• Fixed: Gradient locking bug when no experts exist
• Fixed: Default vram_limit_gb raised from 4.0 to 12.0
• Fixed: Cooldown reset after mitosis

v0.3.0 — Project Cleanup & Definitive API

• Breaking: Removed stale organism.py (use manager.py / OrganismManager)
• New: All public classes exported from top-level dno package
• New: MANIFEST.in ensures README/LICENSE in distributions
• Fixed: train_demo.py updated with correct imports
• Docs: Complete README rewrite with Quickstart, Architecture, API Reference, and Changelog

v0.2.7 — Surgical Training Control

• New: GrowthEngine.inject_layer() — manual mitosis trigger
• New: GrowthEngine.freeze_all_except() — surgical freeze
• New: OrganismManager.get_module() — tag-based lookup
• New: OrganismManager.list_organs() — organ summary
• New: GPTModel blueprint — Transformer Decoder fallback
• New: Robust load_dno() with automatic Zombie Repair
• New: Deep Scan & Repair on load

v0.2.6 — Auto-Casting

• New: Deep scan for nested nn.Embedding layers during auto-cast
• Fixed: LongTensor correctly preserved for Embedding inputs

v0.2.0 — Specialized Organisms

• New: Data-Aware Routing (CPT vs SFT)
• New: Dynamic Gradient Locking
• New: set_specialty() for tagging modules
• New: Smart Aggregation for parallel branches

v0.1.x — Foundation

• Core OrganismManager graph engine
• BaseEvolvableModule with energy tracking
• GrowthEngine with entropy-based mitosis
• SurvivalEngine with KL-Divergence utility monitoring
• IdentityWrapper with gamma gating
• SeedBlock with genetic masking
• Fluid Serialization (.dno format)
• SurvivalEngine garbage collection
• Dashboard visualization

---

🤝 Contributing

DNO is an open-source experiment. We welcome contributions!

• Bug Reports: Open an issue on GitHub
• Feature Ideas: New "Organs" (Memory modules, Attention variants, RL-based growth triggers)
• Pull Requests: Fork, implement, test, and submit!

📜 License

MIT License. See LICENSE for details.