mixle-mlops

An OpenAI-compatible gateway that serves mixle probabilistic models alongside open and hosted LLMs (Llama, DeepSeek, Qwen, …) behind one API — accounts, API keys, a chat UI, multimodal, RAG, an MCP server, and tool calling with a server-side agentic loop. Two things beyond that make it more than an LLM proxy: a probabilistic-bridge stack that lifts a laptop-sized model toward frontier quality on verifiable/structured tasks, and self-evolution that improves served models from their own usage.

It runs end-to-end on a laptop (SQLite + filesystem + a local Ollama) and scales to the cloud (Postgres + object store + Redis) by changing config — no code change. Local-first, cloud-optional.

Quickstart

cp deploy/.env.example deploy/.env          # set MIXLE_SECRET_KEY
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d gateway ollama
docker compose -f deploy/docker-compose.yml --env-file deploy/.env exec ollama ollama pull llama3.2
curl localhost:8000/v1/models               # OpenAI-compatible

Installation

Python 3.10+ (developed and verified on 3.12), on PyPI as mixle-mlops.

pip install mixle-mlops                       # base: gateway, echo model, accounts, local storage
pip install "mixle-mlops[all]"                 # + documents, datasets, export, structured, local, cloud, mcp, image
mixle-mlops init-db && mixle-mlops create-user you@example.com pw --admin
mixle-serve                                    # the gateway on :8000

This installs the gateway package only. The chat UI (frontend/) is a separate Next.js app that isn't shipped on PyPI — clone the repo for that (see Develop, below). See docs/installation.rst for the full extras list.

What's in the box

Documentation

The Sphinx manual starts at docs/index.rst. It includes installation notes, the package map, validation guidance, generated API reference pages, and release-facing guides.

Build it with:

python -m pip install -e ".[docs]"
make -C docs html SPHINXOPTS="-W --keep-going"

Gateway surfaces are documented in docs/gateway-capabilities.rst: substrate serving, telemetry, pool jobs, creation verbs, Mixle model manifests, drift retraining, and LoRA/QLoRA fine-tuning.

Release notes and the current changelog are in docs/release-notes.rst and CHANGELOG.md.

The mixle bridge — frontier quality on a laptop

mixle is the calibrated judge / combiner / router around a small local generator. None of this puts 1T parameters on your laptop; it trades inference-time compute + verification + routing for quality on verifiable / computational / structured / retrieval-grounded tasks (and escalates the rest). All opt-in per request via extra:

Self-evolution — the system improves itself

Served mixle models improve from data and from the platform's own usage, only ever promoting a challenger that verifiably and non-regressively beats the champion (built on mixle.evolve):

• POST /v1/evolve/{model} — run measure → propose → verify → promote (auto-select / online-update / recalibrate / refit), with rollback.
• POST /v1/evolve/tick — one autonomous pass over every hosted mixle model.
• GET /v1/evolve/{model}/signals — the cascade router self-calibrates its threshold from observed traffic.
• Every cascade decision + best-of-N vote is recorded as in-distribution feedback the loop consumes.

Develop

pip install -e ".[dev]"                      # + extras: documents, scale, export, datasets, structured, local,
                                              #           cloud, mcp, image, all
pytest -q                                    # the full suite
ruff check mixle_mlops tests
mixle-mlops create-user you@example.com pw --admin   # a user + an API key
mixle-serve                                  # the gateway on :8000 (or: uvicorn mixle_mlops.gateway.app:app)
cd frontend && npm install && npm run dev    # the chat UI on :3000

Config is env-driven (prefix MIXLE_, see deploy/.env.example and mixle_mlops/config.py). Key knobs: MIXLE_LLM_BASE_URL (Ollama by default), MIXLE_LLM_BACKENDS (per-model local+cloud), MIXLE_DATABASE_URL (sqlite→postgres), MIXLE_OBJECT_STORE_URL (file→s3/gcs/azure/oss), MIXLE_REDIS_URL, MIXLE_REQUIRE_AUTH.

Deploy

mixle-mlops init-cloud aws                    # scaffold a provider-correct .env
# Terraform provisions bucket + Postgres + Redis; Helm runs the same image on any k8s:
cd deploy/terraform/aws && terraform apply
helm install mixle deploy/helm/mixle-mlops -f deploy/helm/mixle-mlops/values-aws.yaml

The same image + chart run on EKS/AKS/GKE/ACK — only three URLs (database / object store / redis) change. Compute is provider-neutral: point MIXLE_LLM_BASE_URL / MIXLE_LLM_BACKENDS at any OpenAI-compatible server running on a GPU VM, marketplace instance, managed Kubernetes cluster, or on-prem host. See deploy/cloud/README.md and deploy/compute/README.md.

Design

The split mirrors mixle's: mixle owns the domain-neutral math (distributions, scoring, calibration, decision, mixle.evolve); mixle-mlops owns serving/orchestration (gateway, accounts, RAG, the bridge components, the evolution worker). The mixle math always upstreams to the core. See docs/overview.rst and docs/package-map.rst for the package boundaries.

Requires mixle.evolve (published on PyPI since mixle 0.6.1; this package pins mixle>=0.6.1) for the self-evolution routes.

Maintainers & contributors

Maintained by Grant Boquet (@gmboquet · grant.boquet@gmail.com).

Contributions, issues, and discussion are welcome — open a PR or an issue.

License

MIT — see LICENSE.