ri-real-intelligence 1.0.0

Real Intelligence Framework – Train intelligent models on any GPU with minimal data

RI - Real Intelligence Framework

Learn from minimal data. Run on any GPU. Build genuine understanding.

RI is a production-grade framework that creates models with "Real Intelligence" – they learn blazingly fast from just 10-100 datasets, run efficiently on any GPU (2GB+), and build genuine understanding through dynamic memory, fact association, logical reasoning, and knowledge graphs. No APIs. No internet required. No massive datasets. Just your data, your hardware, your intelligence.

---

🚀 What Makes RI Different?

Traditional AI (Offline)	RI (Real Intelligence)
Needs thousands of datasets	Learns from 10-100 datasets
Requires expensive GPUs (16GB+)	Runs on any GPU (2GB+)
Days to weeks to train	Minutes to hours
Static frozen weights	Dynamic memory that grows over time
Black box outputs	Transparent fact storage + reasoning chain
No user control over behavior	Properties system (smartness, creativity, etc.)
Pattern matching only	Logical reasoning engine built-in
Full retrain for new data	Incremental retraining with +0.5x multiplier
Command-line only	Beautiful interactive terminal UI with emojis, colors, and progress bars
Single machine	Decentralized platform – share models, API keys, connect to network
No packaging	Portable .rl (Real Learning) file format for sharing models

---

✨ Key Features

🧠 Dynamic Memory

Stores facts, associations, and knowledge as differentiable key-value vectors with adaptive read/write control. Multi-head attention over 1024 memory slots. Learned write gates, forget gates, and access frequency tracking. Memories persist across training sessions. The memory system actively tracks which knowledge is accessed most frequently and consolidates important facts over time.

⚡ Fuins Training System

Novel training units that replace traditional epochs. 10 Fuins with PDS control = intelligent scheduling of data passes. Built-in DIPAW (Dynamic Improvement Percentage Auto-Watch) for automatic early stopping when learning plateaus. Retrain multiplier adds +0.5x per session, meaning your model gets smarter every time you train it.

🎨 Beautiful Interactive Training Interface

Training isn't just functional – it's a joy to watch. RI features a fully interactive terminal dashboard with:

• ANSI color-coded output – purple headers, cyan progress bars, gradient effects
• Emoji indicators – 🧠 for sessions, 🟢🟡🟠🔴 for earning levels, 🏆 for best results, 🎉 for completion
• Real-time progress bar – ████████░░░░░░░░ 62% that fills as training progresses
• Live statistics – tokens/second, elapsed time, earning values updating every few seconds
• One-line summaries – each training step shows earning, speed, progress, and ETA in a single beautiful line
• DIPAW status – clearly shows when early stopping is active or disabled
• Retrain counter – tracks how many times you've trained, with multiplier display

This isn't a dry log file. It's a living dashboard that makes you feel connected to your model's learning process.

🎯 Minimal Data Requirements

Learn English and factual knowledge from just 10-100 text files. Character-level BPE tokenizer builds vocabulary directly from your data. No pretrained embeddings needed. Trained from scratch every time. Add a single text file with new facts and retrain – RI picks it up immediately.

🖥️ Universal GPU Support

Automatic VRAM detection and optimization. Runs on GPUs from 2GB (GTX 1050) to 24GB+ (RTX 4090). Mixed precision (AMP), gradient checkpointing, and dynamic batch sizing ensure OOM-free training on any hardware. CPU-only mode available for machines without GPUs.

🔍 Production-Grade Neural Architecture

• Rotary Positional Embeddings (RoPE) for sequence-aware attention
• Gated Linear Units (GLU) for improved gradient flow
• Multi-head Self-Attention (4 heads × 64 dims)
• 4 Transformer layers with pre-norm residuals
• 43 million trainable parameters
• Gradient checkpointing for memory efficiency
• Advanced Dynamic Memory with multi-head access, write/forget gates, and EMA tracking

🧠 Knowledge Graph + Reasoning Engine

Structured entity storage with category hierarchies, property inheritance, and incompatibility detection. Multi-step logical reasoning with syllogisms, deduction, induction, abduction, analogy, and counterfactual analysis. Verifies generated text against known facts. Automatically detects contradictions like "cats are fish" and explains why they're wrong.

🎛️ Properties System

Full control over RI's intelligence profile. Adjust any of these 15+ parameters from 0-100%:

• Core Intelligence: Smartness, Creativity, Logic, Understanding, Meaning
• Personality: Flaw Tolerance, Randomness, Verbosity, Confidence
• Values: Helpfulness, Honesty, Caution, Curiosity
• Learning: Retention, Adaptability, Skepticism, Fact Adherence
• 7 built-in presets: Genius, Scientist, Teacher, Philosopher, Creative Artist, Assistant, Child

Properties directly affect generation temperature, top-k sampling, and max tokens. Set creativity to 90 and randomness to 70, and RI becomes wildly creative. Set scientist preset and it becomes precise and cautious.

📊 Fact Extraction & Verification

Automatically extracts IS_A, HAS, CAN, and causal relationships from raw text. Knowledge graph with incompatibility detection prevents logical contradictions. Truth verification runs during generation to catch and correct factual errors before output reaches you.

🌍 Multilingual Support

Statistical language detection for 50+ languages using character n-gram profiles. Language-specific stop words and script detection (Latin, Cyrillic, CJK, Arabic, Devanagari, Thai). Train RI in any language by providing text files in that language.

🌐 Decentralized Platform & API System

RI isn't just a local tool – it's a platform. The new decentralized architecture lets you:

• Run your own RI server on your hardware with a single command
• Generate API keys for users to access your model remotely
• Connect to the RI network and share your model with the world
• Package models as .rl files (Real Learning format) for portable sharing
• Track usage statistics – see who's using your model and how much
• Manage permissions – grant generate, memory, or admin access per key
• Revoke access anytime with a single command

Creators run models on their own GPUs. Users access them through API keys. No centralized servers. No monthly fees. Pure peer-to-peer intelligence sharing.

📦 .rl File Format

Portable model packaging format. A single .rl file contains:

• Trained model weights + memory state
• BPE tokenizer vocabulary and merge rules
• Knowledge graph with all entities and facts
• Properties intelligence profile
• Model architecture configuration
• Creator metadata (name, description, tags)

Share a .rl file and anyone can load your model with full capabilities intact. Like a .pt checkpoint but with everything bundled.

🔌 Plugin System

Extensible architecture for custom data sources, text processors, and training callbacks. Auto-discovery of plugins, enable/disable management, event broadcasting. Write a plugin once and use it across all your RI models.

📦 Model Export

Export trained models to TorchScript, ONNX, quantized (INT8/FP16), portable checkpoints, and mobile-optimized formats. GGML-style flat binary export available for C/C++ inference engines.

🌐 HTTP API Server

Serve RI models via REST API with Flask integration. Full endpoint suite:

• POST /generate – Text generation with API key auth
• POST /memory – Memory queries
• GET /status – Server status and GPU stats
• GET /health – Health check
• GET /model-info – Model metadata
• POST /platform/keys – API key management
• GET /platform/stats – Usage statistics

Production-ready with request tracking, API key validation, and connection management.

📈 Real-time Training Monitor

Colored emoji dashboard with progress bars, earning indicators (🟢🟡🟠🔴), tokens/second, GPU utilization, and ETA. ANSI color-coded logging throughout the entire framework.

---

📦 Installation

pip install ri-real-intelligence
pip install ri-real-intelligence[all]      # All features
pip install ri-real-intelligence[dev]      # Development tools
pip install ri-real-intelligence[serve]    # HTTP server + platform
Requirements: Python 3.8+ | PyTorch 2.0+ | CUDA-capable GPU (optional, CPU supported)

🏃 Quick Start
bash
# 1. Add knowledge
ri add-data --text "Cats are small carnivorous mammals. They purr when happy."

# 2. Train with beautiful interactive dashboard
ri train --fuins 10 --pds 60

# 3. Generate with automatic reasoning
ri generate --prompt "Cats are" --max-tokens 100

# 4. Ask questions with knowledge graph
ri answer --question "What are cats?"

# 5. Analyze text for contradictions
ri understand --text "Cats are web development fish"

# 6. Control intelligence profile
ri properties-set --preset scientist
ri properties-set --creativity 90 --randomness 70

# 7. Package your model
ri package --name "Cat Expert" --creator "YourName" --description "Knows everything about cats"

# 8. Start API server
ri serve --port 5000

# 9. Generate API keys for users
ri generate-key --user-id "friend123" --permissions "generate,memory"

# 10. Connect to RI platform
ri connect
🏗️ Full Architecture
text
RI Framework (33 modules)
├── Core System
│   ├── ri_config.py              # Fuins, DIPAW, PDS configuration
│   ├── ri_data.py                # BPE Tokenizer + dataset streaming
│   ├── ri_model.py               # RoPE + GLU + Dynamic Memory (43M params)
│   ├── ri_trainer.py             # Interactive colored emoji training dashboard
│   └── ri_inference.py           # Text generation engine
│
├── Intelligence Layer
│   ├── ri_reasoner.py            # Multi-step logical reasoning engine
│   ├── ri_knowledge_graph.py     # Entity storage with category hierarchies
│   ├── ri_fact_verifier.py       # Truth verification + contradiction detection
│   ├── ri_fact_extractor.py      # IS_A, HAS, CAN pattern extraction
│   ├── ri_fact_trainer.py        # Fact extraction during training
│   ├── ri_understanding.py       # Complete understanding engine
│   ├── ri_grounded_inference.py  # Fact-checked generation
│   └── ri_properties_inference.py # Properties-aware generation
│
├── User Control
│   ├── ri_properties.py          # Intelligence profile management (15+ params)
│   ├── ri_plugin.py              # Extensible plugin architecture
│   └── ri_multilingual.py        # 50+ language detection
│
├── Platform & Sharing
│   ├── ri_platform.py            # Decentralized platform connector
│   ├── ri_packager.py            # .rl file format packager
│   ├── ri_connect.py             # Network connection commands
│   └── ri_serve.py               # Flask HTTP API server with auth
│
├── Data & Processing
│   ├── ri_data_pipeline.py       # Preprocessing, augmentation, versioning
│   ├── ri_vram_tuner.py          # GPU memory optimization
│   └── ri_advanced_memory.py     # Long-term memory consolidation
│
├── Operations
│   ├── ri_benchmark.py           # Performance testing suite
│   ├── ri_export.py              # TorchScript, ONNX, mobile export
│   ├── ri_monitor.py             # Real-time GPU + training dashboard
│   ├── ri_checkpoint.py          # Versioned checkpoint management
│   └── ri_distributed.py         # Multi-GPU distributed training
│
├── CLI Layer
│   ├── ri_cli.py                 # Main entry point
│   ├── ri_commands.py            # All command implementations
│   └── ri_imports.py             # Centralized imports
│
└── Utilities
    ├── ri_utils.py               # Logging, validation, helpers
    └── ri_test.py                # Comprehensive test suite
🎮 Key Concepts
Concept	Description
Fuins	Training units replacing epochs. 10 Fuins at PDS 60 = intelligent data scheduling. Retrain multiplier adds +0.5x per session
DIPAW	Dynamic Improvement Percentage Auto-Watch. Auto-stops when improvement drops below threshold. Set to 0 to disable
PDS	Processing Data Speed (0-100). Lower = higher quality, more passes. Higher = faster but less thorough
Earnings	Lower is better. 🟢 <1.0 (excellent)	🟡 <3.0 (good)	🟠 <6.0 (learning)	🔴 >6.0 (early)
Ones	What traditional AI calls "epochs" – renamed to match RI's unique identity
Dynamic Memory	1024 slots × 128 dims with multi-head attention, write/forget gates, and EMA access tracking
Knowledge Graph	Entities with categories, properties, incompatibility rules, and relationship inheritance
Properties	15+ adjustable intelligence parameters affecting generation behavior in real-time
.rl Files	Real Learning portable model format – complete model package in a single file
API Keys	Per-user access tokens with granular permissions for decentralized model sharing
Platform	Peer-to-peer network where creators run models on their hardware and share via API
⚖️ Honest Tradeoffs
RI Strengths	RI Limitations
Trains in minutes on consumer GPUs	Needs retraining for entirely new domains
Learns from 10-100 files	Output quality scales directly with data quantity
Built-in reasoning + fact checking	Reasoning and neural generation are separate systems
Full control via 15+ Properties	43M params can't match GPT-4's breadth
100% offline and private	Character-level tokenizer limits vocabulary flexibility
Incremental retraining saves time	No vision/audio modalities yet
Knowledge graph prevents nonsense	Graph must be populated for fact checking to work
40,000 dataset capacity	Single GPU focus (distributed mode is basic)
Decentralized platform for sharing	Requires creators to run their own hardware
Beautiful interactive training UI	Terminal-based (no GUI yet)
.rl portable model format	Models are architecture-specific
RI is best for: Domain-specific knowledge engines, private data applications, edge devices, rapid prototyping, education, research, and anyone who wants full control over their AI with zero dependencies on external APIs.

RI is not for: General conversation across all topics, creative writing without domain training, multimodal tasks, or replacing massive language models in every scenario. RI excels at depth, not breadth.

📊 Real Benchmarks
Tested on NVIDIA GTX 1650 (4GB VRAM) with 17 text files (~15KB total):

Metric	Value
Model parameters	43,371,858
Training time (30 Fuins, PDS 35)	~10 minutes
Tokens/second during training	~245
VRAM usage	~2.2GB / 4.0GB
Final earning	1.97 (🟡 yellow)
Retrains to fluent output	3-5 sessions
GPU utilization	82%
Vocabulary size	1,065 BPE tokens
Scaling projections (estimated for same GTX 1650):

Dataset Size	Training Time	Expected Earning
100 files	~30 minutes	<1.0 (🟢)
500 files	~2-3 hours	<0.5
1,000 files	~4-6 hours	<0.3
10,000 files	~1-2 days	<0.1
🔧 Command Reference
bash
# Training & Generation
ri train              # Train a new model with interactive dashboard
ri generate           # Generate text with automatic reasoning
ri add-data           # Add .txt or .json training data
ri memory             # Query model's dynamic memory
ri status             # Show model status, GPU info, properties

# Intelligence & Reasoning
ri reason             # Step-by-step logical reasoning
ri understand         # Analyze text for facts and contradictions
ri answer             # Answer questions using knowledge graph

# Properties & Control
ri properties         # Show current intelligence profile
ri properties-set     # Set properties (smartness, creativity, etc.)
ri properties-presets # List available presets

# Platform & Sharing
ri connect            # Connect model to RI network
ri generate-key       # Create API key for a user
ri list-keys          # List all API keys
ri revoke-key         # Revoke an API key
ri package            # Package model as .rl file
ri model-info         # Show .rl file metadata
ri serve              # Start HTTP API server

# Data Management
ri list-data          # List all training data files
ri delete-data        # Delete a specific data file

# Advanced
ri benchmark          # Run performance benchmarks
ri export             # Export to ONNX/TorchScript
ri monitor            # GPU monitoring dashboard
📄 License
MIT License – see LICENSE file for details.

🙏 Credits
RI concept and design by the creator. Code implementation generated with DeepSeek AI. Built to democratize intelligence – making it possible for anyone with a basic GPU to create, train, share, and deploy models with genuine understanding. No massive datasets required. No expensive hardware needed. No cloud dependencies.

RI – Real Intelligence for everyone. 🧠