outo-llms

Deploy local LLMs behind your own managed, Anthropic-compatible API server.

vLLM or llama.cpp runs in an isolated environment managed by outo-llms, kept apart from the Python environments you already have. Users sign up with a username and password, receive both an outo_st_ session token for account management and an outo_sk_ API key for inference, organize work into workspaces, and every request is metered per workspace. Open the built-in web GUI to sign up or log in, manage workspaces and API keys, and inspect server status, then point your Anthropic SDK (or curl) at the URL for an HTTP call.

Full documentation lives in docs/.

Principles

Fragmentation - small, single-purpose modules.
Fluidity - features can be added or removed as needs change.
Simplicity - a handful of commands and a small API cover the common workflow.
Automation - one guided command gets you a working deployment.
Explicitness - every automated action is announced and logged; the system is never touched without your knowledge.

Features

Isolated engine virtualenvs. vLLM, llama.cpp, and SGLang install into their own virtual environments, never into the system interpreter.
Choice of engines. Bring Hugging Face models through vLLM or SGLang, or GGUF models through llama.cpp, and switch between them with one command.
Accounts with passwords and sessions. Signup takes a username and password, creates a default workspace, and returns both an outo_st_ session token (for management, 14-day expiry) and an outo_sk_ API key (for inference). Login re-issues a session token from the same credentials.
Per-workspace usage metering. Token usage is recorded by workspace and surfaced at GET /v1/usage, with an optional ?workspace= query parameter to scope to one workspace.
Per-key and per-workspace token limits. Set independent token budgets on each API key and each workspace across four periods: total (all time), monthly (UTC calendar month), weekly (ISO week, UTC), and five_hour (rolling 5-hour window). Both the key's own limits and its workspace's limits are enforced; exceeding either returns a 429 error. Configure with POST /v1/workspaces/{name}/keys, PATCH /v1/workspaces/{name}/keys/{key_id}, PATCH /v1/workspaces/{name}, or the outo-llms limits command group.
Anthropic-compatible proxy. POST /v1/messages (plus POST /v1/messages/count_tokens) is the primary inference surface, speaking the Anthropic Messages API shape; the OpenAI endpoints POST /v1/chat/completions and POST /v1/completions are kept for compatibility, and GET /v1/models lists registered models. API keys authenticate the inference endpoints; session tokens authenticate management and usage.
Optional HTTPS. A local outo-llms CA is created under data/certs/ and signs the server certificate. Setup can install the CA into the system trust store so clients trust it without warnings.
Explicit automation with an action log. Every setup step is announced, confirmed when destructive, and appended to actions.log.
Built-in web GUI. Visit the root URL for signup and login, a read-only model catalog, workspace and API-key management, and server status. Visit /docs for the full OpenAPI explorer.
Pre-downloaded weights. outo-llms models add fetches the model's weights into the shared Hugging Face cache immediately, so the first inference request only has to start the engine instead of downloading the model too.

Install

uv tool install outo-llms

Or with pip / pipx:

pip install outo-llms
pipx install outo-llms

Quickstart

outo-llms setup defaults to a public HTTPS deployment on port 443: the API server binds 0.0.0.0:443, a server certificate signed by the local outo-llms CA covers the auto-detected server IP, the CA is installed into the system trust store, and the firewall port is opened on the supported Linux toolchain. Substitute <your-server-ip-or-domain> with the address printed by outo-llms status (the example below uses the documentation placeholder 203.0.113.10):

After setup, open https://<your-server-ip-or-domain>/ in a browser. The built-in web GUI supports signup and login, shows a read-only model catalog, manages workspaces and API keys, and reports server status. The API flow below uses curl instead, and model registration remains CLI-only.

outo-llms setup                              # interactive, fully explicit setup
curl -s -X POST https://<your-server-ip-or-domain>/v1/account/signup \
  -H 'Content-Type: application/json' \
  -d '{"username": "me", "password": "..."}' # returns api_key + session_token + default workspace
outo-llms models add tinyllama               # register a model and download its weights
curl -s https://<your-server-ip-or-domain>/v1/messages \
  -H "Authorization: Bearer outo_sk_..." \
  -H 'Content-Type: application/json' \
  -d '{"model": "tinyllama", "max_tokens": 1024, "messages": [{"role": "user", "content": "hi"}]}'
curl -s https://<your-server-ip-or-domain>/v1/usage \
  -H "Authorization: Bearer outo_st_..."    # per-workspace token accounting (session token)

The CA installed by setup is trusted on the server itself, so plain curl https://... works there. Other machines need to install data/certs/ca.crt once; until they do, keep -k. The setup wizard prints the exact base URL it configured, the path to the action log, and writes a short summary you can copy into your shell history.

Commands

Command	Purpose
`outo-llms setup`	Automated, explicit server setup (engine, HTTPS, firewall, launch)
`outo-llms models add <name>`	Register a model and download its weights
`outo-llms models download <name>`	(Re)download weights for a registered model
`outo-llms models list`	Show every registered model
`outo-llms models remove <name>`	Unregister a model (asks first)
`outo-llms engine list`	List known engines and their installed state
`outo-llms engine use <name>`	Switch the active engine (llamacpp, vllm, or sglang)
`outo-llms engine install [name]`	Install an engine into its isolated virtualenv
`outo-llms engine status`	Show the active engine's runtime status
`outo-llms limits key <user>/<workspace> <key_id> [--set ...] [--clear ...]`	Show or update an API key's token limits
`outo-llms limits workspace <user>/<workspace> [--set ...] [--clear ...]`	Show or update a workspace's token limits
`outo-llms start`	Start the API server in the background
`outo-llms stop`	Stop the background API server
`outo-llms restart`	Restart the API server
`outo-llms status`	Show server, engine, and path status
`outo-llms reset`	Wipe everything back to factory state (asks twice)
`outo-llms version`	Print the version

How it works

The outo-llms server (python -m outo_llms.server) binds 0.0.0.0:443 with HTTPS by default and exposes an Anthropic-compatible API at https://<your-server-ip-or-domain>/. Engines are internal services that bind their own loopback ports (llama.cpp on 8612, vLLM on 8613, SGLang on 8614); clients never reach them directly.

When a request hits POST /v1/messages (or the OpenAI compatibility endpoints), the server authenticates the caller with the outo_sk_ API key, looks up the requested model in the registry, asks the engine manager to ensure the right engine is running with that model, then forwards the request. The active engine streams or returns its response unchanged, and the server records prompt and completion tokens against the calling workspace. Account and workspace management requests use the outo_st_ session token from the same user.

Every state change, install step, and external command goes through outo_llms.core.consent, which announces what is about to happen, asks for confirmation on destructive actions, and writes a timestamped line to logs/actions.log. The built-in web GUI at the root URL and Swagger UI at /docs round out the picture.

Documentation

docs/index.md — entry point for installation, API, configuration, operations, and testing guides.

License

Apache License 2.0. See LICENSE.

Metadata

Release files for outo-llms 0.10.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for outo-llms 0.10.3
File	Size	Uploaded	Provenance
outo_llms-0.10.3.tar.gz	131.6 kB	Oct 21, 2026	With provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for outo-llms 0.10.3
File	Interpreter	ABI	Platform	Reset
outo_llms-0.10.3-py3-none-any.whl 110.9 kB Oct 21, 2026 With provenance	Python 3	none	any	Details

Total release size: 242.6 kB

Release files / outo_llms-0.10.3.tar.gz

Download URL	outo_llms-0.10.3.tar.gz
Size	131.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`465e9aff879f1fb488bf9e8762f837e1afb095c1c5716462c120c4f533c99694`
BLAKE2b-256 checksum How to use checksums	`dc041938821eb4e0f1f25992f82dff63db41256f2f08677fac699eb0c47d0ae0`
Upload date	Oct 21, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/llaa33219/outo-llms
Publishing commit: github.com/llaa33219/outo-llms/tree/86ed6842fcba4a9e11a04fa77c21e495509ad36a

Workflow

Publishing configuration: github.com/llaa33219/outo-llms/blob/86ed6842fcba4a9e11a04fa77c21e495509ad36a/.github/workflows/release.yml
Publishing logs: github.com/llaa33219/outo-llms/actions/runs/39620441763/attempts/1

Artifact

Subject: outo_llms-0.10.3.tar.gz
SHA-256 checksum: 465e9aff879f1fb488bf9e8762f837e1afb095c1c5716462c120c4f533c99694 Verifying attestations

Release files / outo_llms-0.10.3-py3-none-any.whl

Download URL	outo_llms-0.10.3-py3-none-any.whl
Size	110.9 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`5201cfd645511c3f87dd30861617faa277d00a2431969fc7dccd00174ecc2ce9`
BLAKE2b-256 checksum How to use checksums	`9b80c24e5ef94ec10aa8f33fbe043578cc9f933f2585695208ac82c89b03c6e8`
Upload date	Oct 21, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/llaa33219/outo-llms
Publishing commit: github.com/llaa33219/outo-llms/tree/86ed6842fcba4a9e11a04fa77c21e495509ad36a

Workflow

Publishing configuration: github.com/llaa33219/outo-llms/blob/86ed6842fcba4a9e11a04fa77c21e495509ad36a/.github/workflows/release.yml
Publishing logs: github.com/llaa33219/outo-llms/actions/runs/39620441763/attempts/1

Artifact

Subject: outo_llms-0.10.3-py3-none-any.whl
SHA-256 checksum: 5201cfd645511c3f87dd30861617faa277d00a2431969fc7dccd00174ecc2ce9 Verifying attestations

Security for outo-llms 0.10.3

Report project as malware

Provenance

Full provenance

All 2 files in this release have been uploaded with provenance metadata.

Attestation for outo_llms-0.10.3.tar.gz

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/llaa33219/outo-llms
Publishing commit: github.com/llaa33219/outo-llms/tree/86ed6842fcba4a9e11a04fa77c21e495509ad36a

Workflow

Publishing configuration: github.com/llaa33219/outo-llms/blob/86ed6842fcba4a9e11a04fa77c21e495509ad36a/.github/workflows/release.yml
Publishing logs: github.com/llaa33219/outo-llms/actions/runs/39620441763/attempts/1

Artifact

Subject: outo_llms-0.10.3.tar.gz
SHA-256 checksum: 465e9aff879f1fb488bf9e8762f837e1afb095c1c5716462c120c4f533c99694 Verifying attestations

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.