clu-runtime 0.7.0

CLU: adaptive AI runtime for local inference, serving, and continual-learning workflows

CLU Runtime

CLU (Continuous Learning Unit) is a public beta AI runtime for adaptive model execution, local serving, quantization, hardware-aware routing, and continual learning experiments.

pip install clu-runtime

import clu

model = clu.optimize(model)

Current Status

This repository is a beta runtime. The codebase includes working compiler, runtime, serving, learning, and hardware-abstraction components, but public performance claims should be treated as benchmark-specific until reproduced on your machine.

See PRODUCTION_READINESS.md for the current release checklist.

What CLU Provides

• One-line optimization entry point: clu.optimize(model)
• PyTorch, ONNX, and HuggingFace-oriented integration paths
• OpenAI-compatible local serving API
• KV-cache, streaming, batching, and LoRA serving scaffolds
• INT8/INT4 quantization utilities
• CPU SIMD kernels and optional native C kernels
• Intel iGPU / universal GPU abstraction layers
• Continual-learning components: EWC, replay memory, task-boundary detection, and forgetting metrics
• Safe fallback behavior: unsupported paths should fall back instead of taking over user code

Quick Start

import torch
import torch.nn as nn
import clu

model = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 64))
model.eval()

optimized = clu.optimize(model)
output = optimized(torch.randn(1, 64))

Server

clu serve --model model.onnx --port 8000

Optional production hardening:

set CLU_API_KEY=change-me
set CLU_RATE_LIMIT_PER_MIN=120
set CLU_CORS_ORIGINS=http://localhost:3000
clu serve --model model.onnx --port 8000

OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="change-me")
response = client.chat.completions.create(
    model="local",
    messages=[{"role": "user", "content": "Hello"}],
)

Development

git clone https://github.com/RedGhost123/clu-runtime-public
cd clu-runtime-public
python -m pip install -e .[dev]
python scripts/release_audit.py
python -m pytest -q

Packaging Smoke Test

python -m pip wheel . -w dist --no-deps

Public Benchmark Policy

CLU includes benchmark scripts under benchmarks/, but speedup depends on model, input shape, dependencies, CPU/GPU, driver, and quantization mode. Use exact raw result files and machine details when publishing numbers.

License

Apache License 2.0. See LICENSE.