dno 0.2.6

Dynamic Neural Organism (DNO): A self-evolving, growing, and pruning neural network framework.

DNO: Dynamic Neural Organism 🧬

"Don't just train a model. Raise an organism."

DNO (Dynamic Neural Organism) is a next-generation PyTorch framework that treats neural networks as living, evolving biological entities. Unlike static architectures (Transformers, CNNs) that are fixed at initialization, a DNO grows, specializes, and adapts in real-time based on the complexity of the data it consumes.

---

📚 Table of Contents

1. Core Philosophy
2. Installation
3. The DNO Lifecycle (User Guide)
   • Phase 1: DNA Configuration
   • Phase 2: Infancy (Scratch Training)
   • Phase 3: Adulthood (Adaptive Growth)
   • Phase 4: Serialization (Save/Load)
4. Advanced Mechanics
5. API Reference

---

🧠 Core Philosophy

Traditional AI is like a Statue: You carve it (define architecture), polish it (train), and it stays that way forever. DNO is like a Tree: You plant a seed (Seed Cortex). If the environment is rich (complex data), it grows branches (Expert Lobes). If a branch is useless, it withers (Pruning).

Key Features (v0.2.x)

• Neurogenesis (Mitosis): The network physically adds new layers when "confused" (High Entropy).
• Specialized Organisms: The brain divides into a "General Core" and "Expert Lobes".
• Dynamic Gradient Locking: Automatically freezes the General Core when training Experts, and vice versa.
• Smart Routing: Routes "Familiar" data to the Core and "Novel" data to Experts.
• Auto-Casting: Safely handles raw Token IDs (LongTensor) by auto-casting to Float32 where needed.

---

📦 Installation

pip install dno

Requires Python 3.8+ and PyTorch.

---

🧬 The DNO Lifecycle

Raising a DNO involves distinct biological phases.

1. DNA Configuration (DnoConfig)

Before birth, you must define the organism's genetic constraints.

from dno.config import DnoConfig

config = DnoConfig(
    # --- Lifecycle Control ---
    training_phase='scratch',   # Start in 'scratch' (Infancy) or 'adaptive' (Adulthood)
    
    # --- Growth Triggers ---
    entropy_threshold=0.6,      # If confusion > 0.6, consider growing
    evolution_cooldown_steps=100, # Wait 100 steps between growth events
    
    # --- Physical Constraints ---
    max_param_count=100_000_000, # Cap size at 100M parameters
    vram_limit_gb=4.0,           # Stop growing if VRAM exceeds 4GB
    
    # --- Architecture Defaults ---
    d_model=128,                 # Hidden dimension size
    n_heads=4                    # Attention heads (if using Transformer blocks)
)

2. Infancy (scratch Training)

Goal: Build a strong generalist core ("Seed Cortex"). Behavior: Growth is DISABLED. The model behaves like a standard, static PyTorch model.

from dno.core.organism import OrganismManager, BaseEvolvableModule
from dno.core.network import DynamicNetwork
import torch.nn as nn

# 1. Initialize
manager = OrganismManager()
network = DynamicNetwork(manager, config)

# 2. Add the Seed Cortex (Your base model)
# Wrap any PyTorch module in BaseEvolvableModule
seed_layer = nn.Sequential(
    nn.Linear(128, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)
seed = BaseEvolvableModule(seed_layer)
seed.dynamic_id = "seed_cortex"
seed.set_specialty("general") # IMPORTANT: Mark as General Core

network.add_layer(seed)

# 3. Train as usual (Standard PyTorch Loop)
# In this phase, NO growth happens.
output = network(input_data)
loss.backward()
optimizer.step()

3. Adulthood (adaptive Growth)

Goal: Adapt to new, complex tasks by growing specialized organs. Behavior: The model monitors its own entropy. If it encounters data it cannot understand (High Entropy), it triggers Mitosis.

from dno.core.growth import GrowthEngine

# 1. Switch Phase
network.config.training_phase = 'adaptive'

# 2. Initialize Growth Engine
growth_engine = GrowthEngine(network, config)

# ... inside training loop ...
outputs = network(inputs)

# The network automatically calculates 'Entropy' (Confusion) during forward pass.
# You can check this history or let the engine handle it.

# 3. Check for Growth Trigger
is_triggered, reason = growth_engine.check_growth_trigger(
    entropy_history=network.get_recent_entropy(), 
    current_step=step
)

if is_triggered:
    print(f"🌟 EPIPHANY! Growing new expert due to: {reason}")
    
    # TRIGGER MITOSIS
    # Clones the 'seed_cortex' to create a new 'expert_coding_v1' module
    growth_engine.mitosis(
        parent_uuid="seed_cortex", 
        optimizer=optimizer, 
        specialty_tag="expert_coding_v1"
    )

What happens during Mitosis?

1. Cloning: The parent layer is duplicated.
2. Mutation: Smart noise is added to the clone's weights to break symmetry.
3. Rewiring: The clone is connected to the same inputs/outputs as the parent.
4. Specialization: The clone is marked as an "Expert".

4. Fluid Serialization

Standard torch.save fails on DNOs because their architecture (topology) changes dynamically. Use .dno format.

# Save everything (Topology + Weights + Config + History)
network.save_dno("my_organism.dno")

# Load it back
new_network = DynamicNetwork.load_dno(
    "my_organism.dno", 
    module_factory=lambda t: nn.Linear(...) # Factory to reconstruct base layers
)

---

🔧 Advanced Mechanics

Dynamic Gradient Locking 🔒

DNO automatically manages requires_grad to prevent "Catastrophic Forgetting".

• Familiar Data (CPT): The system freezes all Expert layers. Only the General Core updates.
• Novel Data (SFT/High Entropy): The system freezes the General Core. Only the Expert layers update. This happens automatically inside network.forward() based on the data_type or entropy detection.

Auto-Casting 🛡️

DNO v0.2.6+ includes intelligent safety rails for input types.

• If you pass raw LongTensor inputs (Token IDs) to a module that expects floats (like Linear), DNO automatically casts them to Float32.
• It performs a Deep Scan to ensure no Embedding layers are hidden inside the module (e.g., inside a nested GPTBlock). If an Embedding is found, data is preserved as Long.

---

🤝 Contributing

DNO is an open-source experiment.

• Bug Reports: Open an issue on GitHub.
• Feature Requests: We are looking for new "Organs" (Memory modules, Attention blocks).

📜 License

MIT License.