🌟 AuraLite AI

Quickstart · AuraLite Studio · Windows app · FAQ · Quantization

AuraLite AI trains, fine-tunes and runs language models on your own computer — from a desktop app, AuraLite Studio, or the command line. Fine-tune Qwen3.5 (2026), Qwen2.5/3, Llama, Mistral or SmolLM2 on your dialogs with QLoRA, even on a 4-core CPU without a GPU; grow a small model of your own from your texts; chat with the result and export it to llama.cpp, Ollama, LM Studio or transformers, or serve it with an OpenAI-compatible API. Everything runs locally in plain PyTorch, and every part of the engine is readable code — the Transformer architecture behind models like GPT-4, from tokenizer to sampler, open to look into.

AuraLite Studio (2.19) checks the computer and recommends a model and settings that fit it, reads your data before anything trains, runs training with a live chart, and explains errors with what to try. Fine-tuning and training from scratch have their own tabs; a wizard sets everything up:

pip install "auralite-ai[studio]"    # AuraLite + PySide6 (Qt 6)
auralite-studio                      # Start · Train · Fine-tune · Chat · Export · Library · Log

On Windows without Python (2.21): download AuraLite-Studio-<version>-windows-x64.zip from the latest release, unzip it and double-click AuraLite Studio.exe — Python, PyTorch and Qt are inside, auralite.exe next to it is the command line (in PowerShell type .\auralite.exe). Guide: docs/windows-app.md.

Guide: docs/studio.md. The classic interface (auralite-gui, tkinter) stays: since 2.26 the Studio has every one of its features, and since 2.25 the Studio's features go into it too — in 2.26 the Start and Fine-tune tabs (docs/classic.md). If the Studio does not start, it says why — and auralite doctor --studio checks it.

⚡ Quickstart

pip install "auralite-ai[serve]"     # Python 3.11–3.14, from PyPI (2.18.0 on); in a clone: pip install -e '.[serve]'
# or a tag straight from GitHub: pip install 'auralite-ai[serve] @ git+https://github.com/AlexanderNyr/AuraLite-AI@v2.26.0'
auralite doctor --data chats.jsonl   # what this computer can train, with the command to run it (2.19)
auralite bench                       # how fast it trains and writes: a table in ~5 minutes (2.22)
auralite bench --finetune D:\models\Qwen3.5-0.8B   # and a fine-tune of a model downloaded before (2.23)
auralite train --text corpus.txt --model model.pt
auralite generate --model model.pt --prompt "Once upon a time" --length 80
auralite serve --model model.pt      # OpenAI-compatible API on http://127.0.0.1:8000
auralite export --model model.pt     # model.pt2 — runs with plain PyTorch (torch.export)
auralite image-train --images photos/ --preset small --model gen.pt   # your own image generator
auralite image-train --images photos/ --type gan --preset tiny --model gan.pt   # or flow / gan
auralite image-generate --model gen.pt -n 16 --out grid.png
auralite image-eval --model gen.pt --images photos/          # quality score (SWD), no pretrained net
auralite tokenizer train --text corpus.txt --vocab-size 4096 --out tok.json   # only the tokenizer
auralite train --text corpus.txt --tokenizer tok.json --model model.pt        # or a tokenizer.json
auralite train --text books/ "notes/*.md" --model model.pt          # folders and globs (2.15)
auralite train --text chats.jsonl --resume model.pt --model chat.pt # fine-tune a chat model (2.15)
auralite chat --model chat.pt                                       # it answers and stops by itself
auralite chat-eval --model chat.pt --data held_out.jsonl            # stop rate, exact replies (2.16)
auralite train --text chats.jsonl --resume model.pt --lora 8 --save-adapter adapter.pt --model lora.pt
auralite lora apply --base model.pt --adapter adapter.pt --merge --out chat2.pt   # LoRA (2.16)
auralite distill --teacher teacher.gguf --prompts questions.txt --out chats.jsonl  # a teacher's dialogs (2.16)
auralite train --data pairs.jsonl --resume chat.pt --method dpo --model dpo.pt     # DPO / SimPO (2.16)
auralite chat --model chat.pt --json-schema person.json --prompt "Anna, 34, Riga"  # valid JSON (2.17)
auralite data prepare corpus/ --out data/corpus --vocab-size 8192    # big data: token shards on disk (2.18)
auralite data prepare hf:HuggingFaceFW/fineweb-edu:sample-10BT --tokenizer hf:HuggingFaceTB/SmolLM2-135M --max-tokens 2000000000 --out data/fineweb
auralite plan --preset 350m --memory 16GB                            # will it fit? (2.18)
auralite train --data data/fineweb --preset 125m --max-tokens 2000000000 --memory-plan auto --autosave run.pt --model big.pt
auralite train --data data/fineweb --resume run.pt --continue-run --model big.pt   # after a crash or a stop (2.18)
auralite import hf:HuggingFaceTB/SmolLM2-135M-Instruct --out smol.pt    # Llama / Mistral / Qwen2 / Qwen3 into AuraLite (2.18.1)
auralite train --data chats.jsonl --resume hf:Qwen/Qwen3.5-0.8B --quantize nf4 --lora 16 --memory-plan auto --model qwen-chat.pt   # Qwen3.5 with QLoRA, also on a CPU (2.23)
auralite export --model qwen-chat.pt --format gguf --quant q8_0 --out qwen-chat.gguf   # llama.cpp / Ollama (or --format hf)
auralite runs                                                        # every run: its status, loss, time (2.20)
auralite runs continue last                                          # a stopped or interrupted run, where it stood
auralite-studio                      # AuraLite Studio, the desktop app (pip install "auralite-ai[studio]", 2.19)
auralite-gui                         # the classic desktop app: also images, quantization, evaluation

from auralite import AuraLiteEngine

engine = AuraLiteEngine()
engine.train(open("corpus.txt", encoding="utf-8").read(), {"epochs": 3})
print(engine.generate("Once upon a time", length=80))

Everything lives in one package, auralite (auralite.engine, auralite.server, auralite.studio, auralite.quantization, auralite.export, …). Extras: [studio] AuraLite Studio (Qt 6), [serve] server, [hf] Hugging Face + LoRA, [export] ONNX, [torchao] future-proof INT8, [gguf] llama.cpp — see Installation below.

⚡ v2.6 Modern Training Stack

Three upgrades straight from 2024–2025 frontier practice:

• QK-norm (use_qk_norm) — per-head RMS normalization of queries/keys before RoPE. Popularized by ViT-22B (Dehghani et al., 2023) and now standard in OLMo-2, Gemma-2/3, Qwen3 and GLM. Caps attention-logit growth and stabilizes training at higher learning rates.
• Muon optimizer (optimizer="muon") — MomentUm Orthogonalized by Newton–Schulz (Keller Jordan et al., 2024; used in Moonlight / Kimi K2 and every current speedrun). The hidden 2-D weight matrices of the transformer body are updated with a Newton–Schulz-orthogonalized momentum step at a shape-aware LR (match_rms_adamw mode transfers AdamW-tuned LRs). Embeddings, norm scales and an untied LM head automatically stay on a companion AdamW group (split_parameters_for_muon). AdamW remains available (optimizer="adamw") and is auto-selected for LoRA runs.
• 2.12: one default recipe, Muon + QK-norm + WSD + early stopping, for train(), the GUI and the CLI alike (train() and the CLI used AdamW without QK-norm). With early stopping on a 100 KB text Muon + QK-norm reached 2.196 bits per byte against 2.308 for AdamW + QK-norm (three seeds); on a megabyte it was 0.3 % better per step and ties at equal time, because a Muon step costs ~21 % more on a CPU. Numbers: docs/training.md § 8. "optimizer": "adamw" gives the fastest steps.
• 2.13: honest validation, full-speed generation on every thread. A new BPE tokenizer is fitted without the validation text (2.12 learned the held-out words too, which made validation up to 9 % optimistic for large vocabularies); an early stop ends with a short learning-rate decay (1.2 % better on tinyshakespeare, three seeds); every generation forward pass runs on one compute thread, so the server and the GUI decode as fast as the main thread. Numbers: docs/training.md § 6, 8, 10 and docs/engine.md § 10.
• WSD scheduler (lr_schedule="wsd", now the default) — Warmup-Stable-Decay trapezoidal schedule (MiniCPM 2024, DeepSeek-V3-style multi-phase training). Linear warmup → constant LR until wsd_stable_ratio (default 80%) → short decay (cosine / linear / sqrt) to wsd_min_lr_ratio. Matches or beats cosine convergence and lets you branch a decay run from any stable-phase checkpoint. Pure cosine remains available via lr_schedule="cosine".

engine.train(text, {
    "use_qk_norm": True,        # QK-norm per attention head
    "optimizer": "muon",        # Muon for hidden matrices
    "muon_lr": 0.02,
    "lr_schedule": "wsd",       # Warmup-Stable-Decay (default)
    "wsd_stable_ratio": 0.8,
    "wsd_decay": "cosine",
    ...
})

🚀 v2.4 Production-Grade Architecture Update

AuraLite now keeps the original educational single-file entry points and adds a production-oriented package layout:

graph TD
  Engine[AuraLiteEngine] --> Torch[ModernTransformer]
  Engine --> GGUF[GGUFBackend]
  Engine --> HF[HFBackend]
  Torch --> Blocks[TransformerBlock]
  Blocks --> Attention[RoPE + GQA + Sliding Window KV Cache]
  Blocks --> FFN[SwiGLU / optional Top-2 MoE]
  Engine --> Quant[Quantization: GPTQ/AWQ/HQQ/FP8]
  Engine --> RAG[RAG: web + local vector store]
  Server[FastAPI OpenAI Server] --> Engine

New high-impact capabilities

• LLaMA-compatible RoPE: rotate_half formulation, exact inverse-frequency formula, and improved Linear / Dynamic-NTK / YaRN scaling.
• Hardened GQA KV-cache: per-layer caches keep unrepeated KV heads, support sliding-window eviction, and optional FP8/INT8 cache storage.
• Research architecture flags: sliding_window, use_moe, num_experts, use_flex_attention, kv_cache_dtype, tie_word_embeddings.
• Untied input/output embeddings (default): the token embedding (read/context) and the LM head (write/classification) are two mathematically distinct layers, kept as independent parameters by default. Sharing one matrix forces it to both cluster synonyms (input) and separate them (output); the opposing gradients partially cancel and slow convergence. Legacy weight tying remains an explicit opt-in via tie_word_embeddings=True, model.tie_weights() / model.untie_weights().
• One package (2.7.0): everything is importable from auralite — from auralite import AuraLiteEngine, auralite.engine.layers, auralite.engine.model, auralite.server, … The flat names of 2.6 (model_engine, server, …) still work inside a source checkout until 3.0, with a DeprecationWarning.
• Serving: auralite/server/openai_server.py exposes /v1/completions, /v1/chat/completions, and /health.
• RAG upgrade: optional persistent vector store, semantic chunking, HyDE query expansion, and citation-style [source: ...] context.
• Quantization upgrade: HQQ, optional FP8, AWQ alpha/clip grid search, Cholesky-stabilized GPTQ.

Example: train with modern options

from auralite import AuraLiteEngine

engine = AuraLiteEngine()
engine.train(text, {
    "tokenizer": "bpe",
    "bpe_vocab_size": 4096,
    "d_model": 768,
    "n_heads": 12,
    "n_kv_heads": 4,
    "n_layers": 12,
    "d_ff": 2048,
    "seq_length": 2048,
    "sliding_window": 1024,
    "rope_scaling": {"type": "yarn", "factor": 4.0, "original_max_position_embeddings": 2048},
    "use_moe": False,
    "use_compile": True,
})
engine.save_model("auralite-v24.pt")

Example: serve OpenAI-compatible API

AURALITE_MODEL=auralite-v24.pt uvicorn auralite.server.openai_server:app --host 0.0.0.0 --port 8000
# or: auralite serve --model auralite-v24.pt

🚀 Key Features

• 🖥️ AuraLite Studio, the desktop app (2.19): a Qt 6 app (auralite-studio) with its own tabs for fine-tuning ready models and training new ones, plus Chat, Export and a live log. It measures the computer (memory, processor, GPU, a one-second speed test) and recommends the base model, the method (QLoRA, LoRA, every weight) and the settings that fit it — with the memory and the time a run will take, multilingual models for data that is not in English; a setup wizard fills everything in. Data is checked before anything trains (kind, size, script, skipped records, too few examples), errors come with what to try (out of memory, gated models, no connection, data formats) and land in a log file. auralite doctor gives the same recommendation in a terminal. Guide: docs/studio.md.
• 📚 A library of runs, models and datasets (2.20): every training run — of the Studio and of auralite train — is kept with its curve, settings, results and files. A stopped, failed or interrupted run (a crash, a power cut) continues exactly where it stood, dialog runs with LoRA and QLoRA included: the Studio saves a checkpoint every 10 minutes and offers to continue after a crash, and a continued run ends with the same weights as one that never stopped. Compare runs (curves and settings side by side), evaluate a chat model on the dialogs its run held out, train a result further; see the models and datasets on the disk and the room they take. In a terminal: auralite runs list/show/compare/continue/delete. Guide: docs/studio.md § The Library.
• 🆕 Images, agent mode and insights in the Studio; Start and Fine-tune in the classic interface (2.26): the Studio's Images tab trains an image generator of your own on a folder of images — diffusion, flow matching or a GAN, with every setting of the classic Image tab, samples, the images and the model saved and the quality measured — and its Chat tab gets the agent mode (the model calls tools — Python, commands, files in a temporary folder, a web search — only the ones you turn on, with limits on the steps and the seconds per tool) and the insights (the reply's token frequencies, one layer's attention map: a head or their average). With that the Studio has everything the classic interface has. The classic interface gets the Studio's Start tab (this computer, the model that fits it, the recent models, an interrupted run continued from its checkpoint) and its Fine-tune tab (the catalog, a Hugging Face model or a file, QLoRA / LoRA / every weight, the estimate, settings files the Studio reads too), and its fine-tunes are recorded in the library with a checkpoint every 10 minutes. The Windows app brings Pillow, so images train there too. Next: the library, the wizard and recorded Train runs in the classic interface (2.27), then a Russian interface. Guides: docs/studio.md, docs/classic.md.
• 🔁 The Studio does what the classic interface does, and the other way round (2.24–2.25): the Studio's Train tab sets everything the classic one did (a shape of your own, a tokenizer file or one trained alone, prepared datasets, several computers, settings files), the Generate tab has every sampling setting, thinking and web snippets, Export adds .pt2, ONNX and TorchScript; 2.25 adds the Model tab (GGUF with llama.cpp's settings, Hugging Face models imported or run by transformers with LoRA adapters, a fine-tune and the Hub), the Quantize tab (INT8, QAT, GPTQ, AWQ, HQQ, FP16/BF16/FP8: benchmark, compare, save, back to the original) and the Evaluate tab (a chat model on held-out dialogs, lm-evaluation-harness). The classic interface stays and gets the Studio's features too: since 2.25 its menu bar (recent models, the dark theme remembered, the benchmark, the speed test, updates) and the memory in use. Guides: docs/studio.md, docs/classic.md (with the table of what is where).
• Qwen3.5 in AuraLite's own engine (2.23): Qwen3.5 0.8B, 2B, 4B and 9B (March 2026, Apache 2.0) import, fine-tune with LoRA and QLoRA on the processor, chat, save and export to a Hugging Face folder or a GGUF file. Their Gated DeltaNet linear attention — three layers of four, a fixed-size state instead of a growing cache — is computed in plain PyTorch: the same logits as transformers' Qwen3.5, and the exported GGUF computes the same in llama.cpp (within 8e-5). The Studio lists Qwen3.5 first and marks the models before it (older); the advisor recommends a Qwen3.5 when one finishes overnight. auralite bench --finetune takes a model already downloaded (a folder or a checkpoint), and so does the Studio's Benchmark dialog. Measured on a Core i5-2400 from 2011 (16 GB, no GPU): Qwen3.5 0.8B fine-tunes with LoRA at 16 tokens/s, two passes over 1,000 dialogs in about 5 hours - a night; since 2.23.1 the memory plan counts the linear attention too. Guide: docs/training.md § 23.
• 🤗 Pretrained models in AuraLite's own engine (2.18.1): auralite import hf:ORG/NAME reads Llama (2/3/3.1/3.2), Mistral, Qwen2/2.5 and Qwen3 checkpoints into a native model — no transformers — with logits bit-identical to transformers (SmolLM2-135M), downloads that resume and are checked against the Hub's sha256. QLoRA without bitsandbytes (--quantize nf4: 4-bit NormalFloat weights with double quantization, LoRA adapters on top) runs on any CPU PyTorch runs on, AVX-only ones included, and on GPUs; 8-bit AdamW (--optimizer adamw8bit) keeps a quarter of AdamW's optimizer memory. The result exports back as a Hugging Face folder (transformers 4.46 and 5.x load it, same logits) or a GGUF file for llama.cpp, Ollama and LM Studio (F16/Q8_0/BF16/F32, the tokenizer and chat template included). Measured runs and honest CPU speeds for 0.5B–8B models: docs/training.md § 21.
• 🧩 JSON mode, structured outputs and tools (2.17): the server speaks OpenAI's response_format (json_object, json_schema) and tools / tool_choice, and Ollama's format and tools — the official openai client works as is, client.chat.completions.parse with a Pydantic model and pydantic_function_tool included (examples/openai_structured.py). A native model writes the answer under a JSON grammar (constrained decoding) that also keeps room for the shortest valid ending: every answer parses and follows the schema, even from an untrained model (200 of 200 per case, where 0 of 50 parsed without the grammar), at no measurable cost in speed. GGUF models get the schema as a llama.cpp grammar; Qwen2.5-0.5B called the right tool or answered in text in 12 of 16 prompts. From Python: engine.generate_json(messages, schema), engine.chat_with_tools(messages, tools) (docs/engine.md § 12).
• 💬 Chat fine-tuning (2.15): a .jsonl file of dialogs (OpenAI messages, ShareGPT, Alpaca or prompt/completion records) fine-tunes the model in memory, a checkpoint (--resume, optionally with --lora) or a new model into a chat model. The dialogs are written with the chat template exactly as generate_chat prompts (ChatML by default), the loss learns only the assistant's replies and the marker that ends them, and the markers become special tokens — so the model answers in the template and stops by itself (300 of 300 replies in the measured runs, docs/training.md § 15). Training data can also be several files, folders and globs: documents are joined with blank lines and validated on whole held-out documents, and one file trains exactly as before (§ 14). 2.16: dialog runs train for every epoch asked for and learn the prompt with a small weight (0.1) — 86 % exact replies on the benchmark (2.15: 76 % and 64 % on two of the three seeds) — LoRA adapts every linear layer at ten times the learning rate (73 % instead of 57 %), merges into the weights and saves to a file of its own (--save-adapter, auralite lora), and auralite chat-eval measures a chat model on held-out dialogs (§ 16). auralite distill has a teacher model (a GGUF file, a llama.cpp server, a Hugging Face model or an AuraLite checkpoint) answer your questions and writes its replies as dialogs — an interrupted run resumes, the teacher's license goes into the metadata (§ 17) — and preference pairs train a chat model with DPO (DPO-Positive by default) or SimPO: on the arithmetic benchmark the chat models' own mistakes, as pairs, raised their exact answers from 67.5 % to 74.7 % (three seeds, § 18).
• 🖼️ From-Scratch Image Generator: the Image tab (in the Studio the Images tab, 2.26), auralite image-train / auralite image-generate and ImageGenEngine train your own image generator on a folder of images — no pretrained weights. 2.14.2: three model types — denoising diffusion, flow matching (rectified flow, the objective of Stable Diffusion 3: a 2.5× lower FID than diffusion at the same budget on MNIST) and a modern GAN after R3GAN (relativistic loss with R1 + R2, ResNet generator/discriminator without normalisation, EMA generator, truncation trick, optional DiffAugment with ADA's adaptive probability; one network call per image) — plus a quality score that needs no pretrained network (auralite image-eval, the patch sliced Wasserstein distance of ProGAN) and progress for every stage on one bar. A compact self-attention U-Net with presets (Tiny/Small/Medium/Large), ε / x0 / v-prediction, linear / cosine / sigmoid noise schedules (rescaled so they end in pure noise for any step count), Min-SNR-γ, EMA with warmup, AMP, gradient accumulation & checkpointing, channels-last on CPU, warmup+cosine LR, h-flip augmentation, seeding, continue-training and autosave. Class-conditional training from sub-folders with classifier-free guidance, a deterministic held-out validation loss with early stopping, and DPM-Solver++(2M) (good samples in 10–20 steps) next to DDIM / DDPM. Live loss/val plot, preview grid, seeded output, save samples (+ grid), .pt / .safetensors checkpoints. 2.10 made a training step 1.5–2.6× faster on CPU and fixed the noise schedule, EMA and continue-training — see docs/image.md. Requires the optional Pillow dependency.
• PyTorch Engine: Professional-grade tensor operations, autograd and optimization.
• GGUF / llama.cpp Inference: Load .gguf models directly (quantized Llama/Mistral/Qwen/etc.) via llama-cpp-python for generation, streaming, batch prompts, Thinking Mode and web-context prompting.
• Modern Transformer Architecture (LLaMA-style): A decoder-only transformer with RMSNorm (pre-norm), RoPE (Rotary Position Embeddings), SwiGLU feed-forward, optional GQA (Grouped-Query Attention), untied input/output embeddings by default (optional weight tying via tie_word_embeddings=True) and a KV-cache for fast generation.
• Flash Attention: Uses PyTorch scaled_dot_product_attention (fused / memory-efficient kernels) instead of a hand-rolled softmax.
• BPE Tokenizer (built-in): A self-contained mini-BPE trained on your corpus (configurable vocab size), switchable to classic character-level tokenization. BPE dramatically improves text quality — the model learns sub-words instead of single letters.
• Validation Split: A held-out fraction of the text is evaluated every epoch, spread over the whole split. Early stop (on by default since 2.12, 3 epochs) ends a run whose val loss stopped improving and keeps the best epoch's weights; since 2.13 it first decays the learning rate for up to two epochs. A new BPE tokenizer never sees the validation text (2.13).
• Ending a run well (2.12): Stop (Ctrl+C in the CLI) finishes with a short learning-rate decay instead of cutting the run at the peak learning rate, a second press stops at once; a Time limit sizes the schedule so the decay ends on time (docs/training.md § 9).
• torch.compile (optional): Compiles the training step before the first batch (16–20 s on CPU with an empty cache) and falls back to eager if that fails. On a 2-thread CPU it no longer beats the eager loop, so measure before relying on it.
• Continue Training: Fine-tune the model currently in memory (trained or loaded) on a new file instead of starting from scratch — on text, or on dialogs to make it a chat model (2.15).
• Autosave: Optional checkpoint autosave every N epochs — never lose a long run.
• Hardware Acceleration: Automatic detection and usage of NVIDIA CUDA (GPU) for training and generation, with a seamless fallback to CPU.
• Full CPU Multithreading: Automatically configures PyTorch (and the OpenMP/MKL backends) to use all available CPU cores. Batches are gathered in the training process (one indexing operation each), so no worker processes are started.
• Mini-Batch Training: Training runs on shuffled mini-batches of non-overlapping windows (configurable Batch Size); one epoch is one pass over the text, with a fresh window offset every epoch.
• Advanced GUI: A comprehensive control panel built with tkinter that allows real-time interaction with the model.
• Hyperparameter Tuning: Full control over the AI's "brain" directly from the interface:
  • Learning Rate: Controls how fast the model adapts to new data.
  • Epochs: Determines how many times the model studies the dataset.
  • Model Dimension (D_Model): Sets the size of the internal vector representations.
  • Feed-Forward Dimension (D_FF): Controls the capacity of the processing layers.
  • Heads (N_Heads) / Layers (N_Layers): Shape of the attention mechanism and network depth.
  • Context Window (Seq Length): Defines how many previous characters the AI considers when predicting the next one.
  • Batch Size: Number of samples processed per optimizer step. Larger values use more memory but better utilize multiple CPU cores / the GPU.
  • Dropout / Grad Clip: Regularization and training-stability controls.
• Sampling Controls: Temperature, Top-K, Top-P (nucleus) and Repetition Penalty for generation.
• Dense Next-Token Loss: The loss is computed over every position of the context window (nanoGPT-style), making training far more sample-efficient than last-position-only prediction.
• Tabbed GUI: Three clean tabs — 🏋️ Training (hyperparameters, tokenizer options, loss history), ✨ Generation (sampling + prompt + output), 💾 Model (save / load + full model info).
• 🧠 Thinking Mode (NEW): Two-pass generation — the model first free-writes a higher-temperature draft ("thoughts"), then the final answer is generated conditioned on that draft (self-conditioning). No retraining needed; works with any existing checkpoint. The GUI shows the thinking block and the final answer separately.
• 🌐 Web Search / mini-RAG (NEW): Optional DuckDuckGo search (no API key, stdlib-only) — top result snippets are injected into the prompt as retrieval context before generation. Can be combined with Thinking Mode. If the search fails (offline), generation gracefully continues without it.
• ⚡ Model Quantization (NEW v2.2): Comprehensive quantization toolkit with 6 methods: Dynamic INT8, Static INT8, QAT, GPTQ (INT2/3/4/8), AWQ (INT4/INT8), Half Precision (FP16/BF16). Dedicated GUI tab with benchmarking, comparison tables, and one-click quantization. Supports packed low-bit storage, Hessian-based rounding, activation-aware weight protection, and fake-quantize STE for training.
• Custom Training: Upload any .txt file to teach the built-in AuraLite .pt model specific styles, languages, or fictional worlds.
• External GGUF Models: Open .gguf files from the Model tab and use them immediately for inference. GGUF files are inference-only in AuraLite; train/fine-tune with AuraLite’s native .pt checkpoints.
• Interruptible Training: Ability to stop training at any point and preserve the learned weights for immediate testing.
• Gradient Checkpointing: Trade compute for memory — enables training of larger models on consumer GPUs. Toggle in the Training tab.
• 💬 Chat / Instruction Mode: Dedicated chat tab with system/user/assistant roles, multiple templates (ChatML, Llama-2, Mistral, Gemma, Phi), conversation history, and real-time token streaming.
• 📏 Long generations stay coherent (2.11): generation attends to the newest seq_length tokens, the distances the model was trained on. Before 2.11 everything past the trained length turned into noise (held-out loss 3.97 against 1.48 at positions 256–511 of a model trained on 64). The KV cache stays bounded and long prompts are prefilled in chunks. --full-context / engine.set_context_window("full") restores the old behaviour — see docs/engine.md.
• 🔄 RoPE scaling (linear / NTK / YaRN): for continued training on longer windows and for imported models. On its own it does not let a model read past the length it was trained on (measured in docs/engine.md § 9); continue_training with a longer seq_length extends a model (docs/training.md § 7).
• 🌙 Dark Theme: Toggle available in the header.
• ☁️ Hugging Face Hub Integration: Push/pull models and LoRA adapters to/from the Hub.
• 📊 Model Evaluation: Built-in evaluation using lm-evaluation-harness (ARC, MMLU, GSM8K, etc.).

✅ Feature status: implemented / limited / opt-in (v2.6.5)

Honest status of the features that are easy to over-promise. Everything marked "implemented" has a regression test in tests/ (see tests/test_improvements_v265.py).

Diagnostics

auralite info --model model.pt --backends        # quantization backend availability
auralite info --model model.pt --bench 128       # tokens/s, weight memory, KV-cache mode
auralite info --model model.pt --quiet           # no progress/library noise
auralite train --text corpus.txt --params p.json  # engine logs go to stdout
AURALITE_LOG_LEVEL=WARNING auralite generate ...  # or silence INFO library lines
AURALITE_AGENT_MAX_ITERATIONS=12 AURALITE_AGENT_TOOL_TIMEOUT=30 auralite-gui   # the agent fields start there (2.26: the Studio too)

📚 Documentation

🛠 Technical Specifications

• Framework: PyTorch (Tensors, Autograd, AMP on CUDA, optional torch.compile).
• Attention: Multi-Head Self-Attention via PyTorch SDPA (Flash / memory-efficient kernels) with Causal Masking, RoPE and optional GQA.
• Normalization: RMSNorm (pre-norm), as used in LLaMA / Mistral / Qwen.
• Optimizer: AdamW (betas 0.9/0.95, weight decay) with cosine LR schedule + linear warmup and gradient clipping.
• Input/Output: Built-in BPE tokenizer (recommended) or character-level tokenization; old char-level checkpoints load transparently.

📦 Installation & Setup

Prerequisites

• Python 3.11 – 3.14. Python 3.10 reached end-of-life on 2026-10-01; AuraLite 2.6.x was the last series that supported it.
• NVIDIA GPU (optional). No CUDA Toolkit is needed — only the driver: on Linux the default torch wheel from PyPI already contains CUDA; on Windows pick the CUDA build on pytorch.org.

Install

From PyPI (2.18.0 on) — the same extras as below:

pip install auralite-ai                 # core: torch, numpy, matplotlib (training, generation, GUI)
pip install "auralite-ai[serve,data]"   # + the server, + big datasets (Hugging Face, Parquet, .zst)

From a clone:

git clone https://github.com/AlexanderNyr/AuraLite-AI.git && cd AuraLite-AI
pip install -e .              # core: torch, numpy, matplotlib (training, generation, GUI)
pip install -e '.[serve]'     # + OpenAI/Ollama-compatible server (FastAPI, uvicorn)
pip install -e '.[hf]'        # + Hugging Face models and LoRA fine-tuning (transformers, peft)
pip install -e '.[qlora]'     # + 4-/8-bit QLoRA loading (bitsandbytes; not on macOS)
pip install -e '.[data]'      # + big datasets: Hugging Face datasets, Parquet, .zst, Rust tokenizers (2.18)
pip install -e '.[test]'      # + the test suite

pip install -r requirements.txt installs the same core dependencies without installing the package (enough for python gui_app.py from the checkout). An installed AuraLite adds two commands: auralite (CLI, also python -m auralite) and auralite-gui (desktop app, also python -m auralite.gui).

.gguf models (llama.cpp)

PyPI only has the source of llama-cpp-python, so a plain pip install compiles llama.cpp — minutes of waiting, and an error on Windows without Visual C++ build tools. Use the prebuilt wheels instead:

# CPU: Linux x86-64/arm64, Windows x64, Apple Silicon
pip install -e '.[gguf]' --only-binary llama-cpp-python \
    --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
# NVIDIA GPU, CUDA 12.5+: Linux x86-64, Windows x64
pip install -e '.[gguf]' --only-binary llama-cpp-python \
    --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu125

--only-binary turns a missing wheel into an immediate error instead of a long source build. For other accelerators see the llama-cpp-python installation guide.

Docker

docker build --target runtime-cpu -t auralite-ai:cpu .
# copy a checkpoint into the data volume (the container user owns /data)
docker run --rm -v auralite-data:/data -v "$PWD":/in:ro auralite-ai:cpu cp /in/model.pt /data/
docker run -p 8000:8000 -v auralite-data:/data -e AURALITE_MODEL=/data/model.pt auralite-ai:cpu

The image serves the OpenAI-compatible API on port 8000. It runs as an unprivileged user and keeps models, caches and state in /data (AURALITE_HOME). --target runtime-cuda builds the GPU variant (docker run --gpus all …). Tagged releases publish ghcr.io/alexandernyr/auralite-ai:<version>.

📖 How to Use

1. Launch the App:

   auralite-gui          # installed (pip install -e . or a wheel)
   python gui_app.py     # straight from a source checkout
   

2. Use an existing .gguf model (optional):
   • Open the 💾 Model tab.
   • Click Load Model and choose a .gguf file.
   • Go to ✨ Generation and prompt it. Streaming, batch mode, Thinking Mode and Web Search work with GGUF too.
   • A GGUF options dialog lets you set n_ctx, GPU layers, CPU threads, n_batch, chat format, chat-completion mode, mmap and mlock.
   • Note: .gguf files are quantized llama.cpp inference models; AuraLite does not train or re-save them as .pt.
   • The same GGUF defaults can be set via environment variables before launch: AURALITE_GGUF_N_CTX, AURALITE_GGUF_N_GPU_LAYERS, AURALITE_GGUF_N_THREADS, AURALITE_GGUF_N_BATCH, AURALITE_GGUF_CHAT_FORMAT, AURALITE_GGUF_USE_CHAT, AURALITE_GGUF_USE_MMAP, AURALITE_GGUF_USE_MLOCK.
3. Configure & Train a native AuraLite model:
   • Adjust the Hyperparameters to suit your hardware and dataset.
   • Click "Select .txt File" and provide your training data.
   • Click "Start Training". Monitor the Loss value; a decreasing loss indicates the AI is learning.
4. Generate Text:
   • Enter a Seed phrase to give the AI a starting point.
   • Set the desired Length of the output.
   • Click "Generate Text" and watch the AI create content based on its training.

🔨 The Windows app (no Python)

Every release since 2.21 attaches AuraLite-Studio-<version>-windows-x64.zip: AuraLite Studio and the auralite command with Python, PyTorch (CPU) and Qt inside — unzip it anywhere and start AuraLite Studio.exe. CI builds and checks it on every push (the windows-app job); the Release workflow builds it from the tag and attaches it with its SHA-256 and signed provenance.

1. To build the same app yourself, run build_exe.bat in a clone (or the commands in docs/windows-app.md).
2. It installs the pinned packages of packaging/app/requirements.txt into .venv-app, installs AuraLite from the folder and runs packaging/app/build_app.py --zip.
3. The app is in dist/app/AuraLite-Studio-<version>/ and the zip in dist/; packaging/app/smoke_app.py checks either the way a user meets it. Distribute the whole folder (or the zip).

⚠️ Hardware Compatibility Note

• CUDA Acceleration: Requires an NVIDIA GPU with Compute Capability 5.0 or higher.
• CPU Fallback: If a compatible GPU is not detected, AuraLite AI automatically switches to CPU mode. While slower, it remains fully functional.
• Memory Tip: For CPU-only users, keeping D_Model at 64 and Seq Length at 16 is recommended for optimal performance.