YURIformer 0.1.0

A vibe-coded, hierarchical dendritic computing framework.

YURIformer :3

heya everyone :] welcome to YURIformer! this entire architecture is aggressively vibe coded, meaning I used AI to help me mostly with the code to prototype faster [and i cant code] but all the things like triton kernels and DEQ solver algorithims are verified by me :3

tho it was designed to be a modular library for efficiency and innovative learning rules, neural nets, and efficient numerical operations :]

---

[Written by a human] (Q&A Time! :3)

What is the purpose of the project?

The purpose of this project was to simplify the implementation of innoative learning rules and architectures :] currently I have implemented DEQ kernels for learning and posit 16 kernels for our DEQs and numeric operations :3

What is under development?

• I will explore neuromorphic computing next like adding OSTL , OSTTP, OTTT, and much more ya can use :3
• I will also explore niche parts of ML beyond just these 2
• While i did mention i will add neural networks they are sadly under progress :[

How/What to contribute?

• while this project IS vibecoded [tho i am learnibng to write code by hand!] I encourage people to contribute using their own sweats and tears [aka written by real human :) ]
• I also encourage opening issues for bugs or errors that i could try to fix well
• I also heavily encourage you to make PRs since we need more human written code more human creativity :]

---

[Generated by an AI]

Installation

Since the project is structured properly with a pyproject.toml, you can easily install it locally. Just clone the repo and install it in editable mode :3

git clone https://github.com/moelanoby/YURIformer.git
cd YURIformer
pip install -e .

This will automatically install dependencies like torch, triton, and numpy, making all the architecture_kernels, numeric_kernels, and learning_rules importable from anywhere in your environment :D

How to Use DEQ Kernels (Drop-in Replacement for Infinite Depth)

We have a fully modular Deep Equilibrium (DEQ) library that replaces Backpropagation Through Time (BPTT) for implicit depth (weight-tied layers), rather than sequence time. It effectively gives you an infinite-depth network with O(1) memory and super fast training! :]

import torch
import torch.nn as nn
from learning_rules.DEQ_kernels import DEQModule, HybridConfig, SolverFactory

# 1. Define your recurrent cell (the exact same one you use for BPTT!)
class MyCell(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.fc = nn.Linear(dim, dim)
        self.norm = nn.LayerNorm(dim)
        
    def forward(self, z, x):
        return torch.tanh(self.norm(self.fc(z) + x))

# 2. Pick a solver config (e.g., Hybrid, Anderson, Broyden, or PJWR) :p
cfg = HybridConfig(max_iter=40, tol=1e-4)

# 3. Wrap your cell in the DEQModule :3
cell = MyCell(dim=64)
solver = SolverFactory.create(cfg, cell)
deq_layer = DEQModule(cell, solver=solver, backward_mode='phantom')

# 4. Forward pass finds the fixed point implicitly! :D
x = torch.randn(32, 64)
z_star = deq_layer(x)

Wanna see it in action? Check out the drop-in replacement example! :D

We wrote a super detailed script that shows you exactly how to transition your codebase from BPTT to DEQ! Go open up examples/deq_dropin_replace_bptt.py and give it a read :3 It has five different patterns showing you:

• The minimal swap (changing just 4 lines of code!)
• How to pick your solver dynamically using config dataclasses
• How to switch between phantom and neumann-1 backward modes
• How to add Jacobian Regularization for extra stability! :p

Seriously, it's fully documented and runnable. Try running python examples/deq_dropin_replace_bptt.py to see the side-by-side performance benchmarks happen live on your machine :]

Benchmarks

We benchmarked our DEQ solvers against standard BPTT on a synthetic task (30 training steps). Using our custom solvers with phantom gradients, we achieve great accuracy with O(1) memory scaling while avoiding the deep unrolling overhead :p

Training Method	Memory Scaling	Speed (30 steps)	Accuracy	Where they break (usually)
Standard BPTT (unroll=5)	O(N)	0.12s	100%	its O(L) memory cost
Standard BPTT (unroll=20)	O(N)	0.11s	92%	its O(L) memory cost
DEQ (PJWR)	O(1)	0.03s	55%	it is fast! but falls short in accuracy if not intialized correctly
DEQ (Broyden)	O(1)	0.19s	55%	without anderson warm up it also falls short in accuracy
DEQ (Hybrid)	O(1)	0.27s	100%	yes it is fast but might fall short in other benchmarks that this doesnt show but not as much as the previous 2
DEQ (Anderson)	O(1)	0.29s	100%	THE INDUSTRY STANDARD but CAN be a bit slower than the hybrid in long runs

Note: The solvers find the fixed point z implicitly, bypassing the need to store activations for the entire depth history! :]*