mixle-mlops 0.7.0

All-in-one AI platform: host mixle models + open LLMs behind one OpenAI-compatible gateway, with accounts, API keys, multimodal, MCP, a chat UI, and a principled (mixle-powered) RLHF feedback loop.

mixle-mlops

An OpenAI-compatible gateway that serves mixle probabilistic models alongside open and hosted LLMs (Llama, DeepSeek, Qwen, …) behind one API — accounts, API keys, a chat UI, multimodal, RAG, an MCP server, and tool calling with a server-side agentic loop. Two things beyond that make it more than an LLM proxy: a probabilistic-bridge stack that lifts a laptop-sized model toward frontier quality on verifiable/structured tasks, and self-evolution that improves served models from their own usage.

It runs end-to-end on a laptop (SQLite + filesystem + a local Ollama) and scales to the cloud (Postgres + object store + Redis) by changing config — no code change. Local-first, cloud-optional.

Quickstart

cp deploy/.env.example deploy/.env          # set MIXLE_SECRET_KEY
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d gateway ollama
docker compose -f deploy/docker-compose.yml --env-file deploy/.env exec ollama ollama pull llama3.2
curl localhost:8000/v1/models               # OpenAI-compatible

Installation

Python 3.10+ (developed and verified on 3.12), on PyPI as mixle-mlops.

pip install mixle-mlops                       # base: gateway, echo model, accounts, local storage
pip install "mixle-mlops[all]"                 # + documents, datasets, export, structured, local, cloud, mcp, image
mixle-mlops init-db && mixle-mlops create-user [e-mail] pw --admin
mixle-serve                                    # the gateway on :8000

This installs the gateway package only. The chat UI (frontend/) is a separate Next.js app that isn't shipped on PyPI — clone the repo for that (see Develop, below). See docs/installation.rst for the full extras list.

What's in the box

Capability	Surface
OpenAI-compatible chat (streaming) over any hosted model	POST /v1/chat/completions, GET /v1/models
Host open LLMs through their standard server (Ollama/vLLM/llama.cpp/hosted)	MIXLE_LLM_BACKENDS (local + cloud in one registry)
Host mixle models with a real probabilistic surface	POST /v1/mixle/{predict,score,latent,decide}
Tool calling + server-side agent loop (executes MCP tools, RAG, mixle decide/predict, exact compute)	extra.agent, /mcp
Accounts, API keys, OAuth (Sign in with Google/Apple)	/auth/*, mk-… keys
Multimodal (image inputs), RAG (PDF/DOCX/PPTX upload + retrieval), image gen, dataset gen	/v1/files, /v1/documents, /v1/rag/search, /v1/images/generations, /v1/datasets
Conversations (persisted threads + json/markdown/pdf export)	/v1/conversations
LoRA/QLoRA fine-tuning (SFT via chat template + prompt masking, or raw continuation; resume a prior adapter) and loading a trained adapter into the live gateway	/v1/fine_tunes, /v1/models/load
Caching + rate limiting (memory / Redis), MCP server	extra flags, MIXLE_REDIS_URL, /mcp
Chat UI (Next.js, assistant-style conversation surface)	frontend/
Multi-cloud deploy (AWS/Azure/GCP/Alicloud)	Helm chart + Terraform + mixle-mlops init-cloud
Universal cloud compute (GPU VMs, marketplaces, managed k8s, on-prem)	Any OpenAI-compatible endpoint + generic Docker/Kubernetes GPU recipes

Documentation

The Sphinx manual starts at docs/index.rst. It includes installation notes, the package map, validation guidance, generated API reference pages, and release-facing guides.

Build it with:

python -m pip install -e ".[docs]"
make -C docs html SPHINXOPTS="-W --keep-going"

Gateway surfaces are documented in docs/gateway-capabilities.rst: substrate serving, telemetry, pool jobs, creation verbs, Mixle model manifests, drift retraining, and LoRA/QLoRA fine-tuning.

Release notes and the current changelog are in docs/release-notes.rst and CHANGELOG.md.

The mixle bridge — frontier quality on a laptop

mixle is the calibrated judge / combiner / router around a small local generator. None of this puts 1T parameters on your laptop; it trades inference-time compute + verification + routing for quality on verifiable / computational / structured / retrieval-grounded tasks (and escalates the rest). All opt-in per request via extra:

Lever	Request	What it does
Best-of-N self-consistency	extra.best_of_n: 5	sample N, return the majority answer + a calibrated confidence (X-Self-Consistency)
Cascade router (FrugalGPT)	extra.cascade: {frontier, threshold, n}	answer locally when confident, escalate to a frontier model only on the hard tail (X-Cascade-Escalated)
Mixture-of-Agents	extra.moa: {proposers, aggregator}	several models propose, an aggregator synthesizes
Program-offload (PAL)	the mixle_solve tool (agent mode)	the model offloads exact arithmetic/probability/stats to a deterministic solver instead of doing mental math

Self-evolution — the system improves itself

Served mixle models improve from data and from the platform's own usage, only ever promoting a challenger that verifiably and non-regressively beats the champion (built on mixle.evolve):

• POST /v1/evolve/{model} — run measure → propose → verify → promote (auto-select / online-update / recalibrate / refit), with rollback.
• POST /v1/evolve/tick — one autonomous pass over every hosted mixle model.
• GET /v1/evolve/{model}/signals — the cascade router self-calibrates its threshold from observed traffic.
• Every cascade decision + best-of-N vote is recorded as in-distribution feedback the loop consumes.

Develop

pip install -e ".[dev]"                      # + extras: documents, scale, export, datasets, structured, local,
                                              #           cloud, mcp, image, all
pytest -q                                    # the full suite
ruff check mixle_mlops tests
mixle-mlops create-user [e-mail] pw --admin   # a user + an API key
mixle-serve                                  # the gateway on :8000 (or: uvicorn mixle_mlops.gateway.app:app)
cd frontend && npm install && npm run dev    # the chat UI on :3000

Config is env-driven (prefix MIXLE_, see deploy/.env.example and mixle_mlops/config.py). Key knobs: MIXLE_LLM_BASE_URL (Ollama by default), MIXLE_LLM_BACKENDS (per-model local+cloud), MIXLE_DATABASE_URL (sqlite→postgres), MIXLE_OBJECT_STORE_URL (file→s3/gcs/azure/oss), MIXLE_REDIS_URL, MIXLE_REQUIRE_AUTH.

Deploy

mixle-mlops init-cloud aws                    # scaffold a provider-correct .env
# Terraform provisions bucket + Postgres + Redis; Helm runs the same image on any k8s:
cd deploy/terraform/aws && terraform apply
helm install mixle deploy/helm/mixle-mlops -f deploy/helm/mixle-mlops/values-aws.yaml

The same image + chart run on EKS/AKS/GKE/ACK — only three URLs (database / object store / redis) change. Compute is provider-neutral: point MIXLE_LLM_BASE_URL / MIXLE_LLM_BACKENDS at any OpenAI-compatible server running on a GPU VM, marketplace instance, managed Kubernetes cluster, or on-prem host. See deploy/cloud/README.md and deploy/compute/README.md.

Design

The split mirrors mixle's: mixle owns the domain-neutral math (distributions, scoring, calibration, decision, mixle.evolve); mixle-mlops owns serving/orchestration (gateway, accounts, RAG, the bridge components, the evolution worker). The mixle math always upstreams to the core. See docs/overview.rst and docs/package-map.rst for the package boundaries.

Requires mixle.evolve (published on PyPI since mixle 0.6.1; this package pins mixle>=0.6.1) for the self-evolution routes.

Maintainers & contributors

Maintained by Grant Boquet (@gmboquet · [e-mail]).

Contributions, issues, and discussion are welcome — open a PR or an issue.

License

MIT — see LICENSE.