mcp-modelmanager

Manage your own local model machine through a language-model tool (Claude, ChatGPT, etc.). The models and their data stay on your hardware; the server is designed so the machine need not be opened to the internet by this software.

This MCP server gives an AI session controlled access to your own Ollama/model machine: read state, pull and remove models, switch the inference service to a different model, build custom Ollama variants and run LoRA training. The models run on your hardware. The server process itself talks only to your own computer or local network, never to a public address, on its own. What it does knowingly reach out to a public address for is a model: calling fetch_model makes the model machine connect to Hugging Face at runtime to download it. That is a deliberate, tool-triggered outbound connection, not something the server does behind your back or only once at setup. switch_service never reaches Hugging Face itself; if the model is not yet on the machine, it refuses and tells you to run fetch_model first.

What that does and does not promise. The inference itself stays local: your prompts to the managed models and their answers do not go to any cloud model provider. But the language model you drive this server with (Claude, ChatGPT, etc.) is itself a hosted service in the usual case. Your instructions to it and the tool results it sees pass through that provider like any other MCP call. "Local" here means the managed machine and its data; it is not a claim that nothing you type ever reaches a third party. Choose the driving model accordingly, or drive it with a local one.

Operating systems: The plain query path (reading status and models over the tunnel) runs cross-platform. Management and setup (service and training control) require a POSIX system with systemd (systemd user unit, ssh -F /dev/null); Windows is not supported for management.

The server ships with no address of its own and no default values for a machine. You enter where your machine is; without those details the server deliberately does not start (see Required fields and SETUP.md).

What the server can and cannot do

It can: query state (GPU memory, disk space, running service, containers), list/pull/remove models, switch the vLLM inference service to a different model (with an up-front estimate of the memory need), build custom Ollama variants (fixed system prompt, parameters, later an adapter) and start/track/cancel LoRA training.

It cannot and deliberately will not: run an arbitrary command on the machine. There is no tool for free remote control. Every operation is a fixed template; every parameter first passes an allowlist (see Security model).

It is not a query bridge. For merely asking a model you use a separate, read-only MCP server (e.g. a simple HTTP bridge to Ollama). This server manages the machine. Two separate servers ensure that a query call never accidentally holds management rights.

Your machine: three possible cases

The model machine can be attached in different ways. All three cases are supported; you only enter a different address.

Case	Where Ollama/vLLM runs	What you enter as MM_VM_HOST
a) same machine	on the same computer as this server	127.0.0.1
b) on the local network	on another computer in the same LAN	its LAN address, e.g. 192.168.1.10
c) over tunnel/VPN	on a remote computer, reachable via VPN/tunnel	the address at your own/VPN end, e.g. 127.0.0.1 for a local SSH tunnel, or the VPN address

Safeguard: the server process's own MCP/SSH/HTTP traffic goes only to the own computer or the local network. Addresses outside (public IPs, foreign hostnames) are rejected as MM_VM_HOST, so the server does not reach out to a public address on its own to talk to your model machine. For case c) this means: the tunnel/VPN ends locally or in a private address range, and you enter that local/private address, not the public address of the far side. This safeguard does not cover model downloads: fetch_model makes the model machine itself reach out to Hugging Face, a public address, whenever you ask it to fetch a model (see the caveat above); switch_service refuses a model that is not already on the machine instead of fetching it. fetch_model(source="ollama") sends the machine to a second, equally deliberate public address: it makes the machine's Ollama daemon pull the model via /api/pull, which reaches out to registry.ollama.ai for plain model names; a prefixed name such as hf.co/user/repo sends it to that other registry instead.

The plain query (state, version, loaded models) runs over HTTP to the service port. Management (pulling models, switching the service, training) runs over a secured SSH control channel. For management you therefore need SSH access to the machine, also in case a) (then SSH to 127.0.0.1).

Installation

Requirement: Python 3.11 or newer.

With uv (recommended):

uv venv --seed .venv
.venv/bin/pip install -r requirements.txt

--seed ensures pip is present in the venv.

With Python built-ins (fallback):

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

Should python3 -m venv produce a venv without pip on your system (missing ensurepip), use the uv path above.

There are only two direct dependencies, both pinned: the MCP SDK (mcp==1.29.0) and an HTTP client (httpx==0.28.1). These two pins live in requirements.txt and pyproject.toml; the full resolved dependency tree (including the transitive packages anyio, pydantic, starlette, uvicorn, jsonschema, pyjwt) is additionally pinned in uv.lock. CI checks with uv lock --check that the lock still matches pyproject.toml, then installs the exact locked tree for the tests, so a drifted lock fails CI. A separate, advisory pip-audit job checks the locked tree for known vulnerabilities on every push to main and pull request and again weekly; it does not block a pull request. The SSH access uses the system's ssh program.

Required fields

The server reads its configuration exclusively from environment variables. These three you must set, otherwise it does not start:

Field	Meaning
MM_VM_HOST	Address of your model machine (127.0.0.1, LAN IP or tunnel/VPN address)
MM_VM_USER	SSH user name on the machine (only for management over the control channel)
MM_CONTAINER_ROOT	Root directory on your model machine under which all model, adapter and cache data live

If one is missing, the start aborts with a clear message that says which field is missing. Example:

$ PYTHONPATH=src .venv/bin/python -m mcp_modelmanager.server
MM_CONTAINER_ROOT is not set. Enter the root directory on your model machine
under which all model, adapter and cache data live. Without this value the
server does not start. See SETUP.md.

All further fields have sensible defaults (Ollama and vLLM standard ports, key storage in your own ~/.ssh) and are only set when needed; see Environment variables.

The full step-by-step setup from the empty state to the running server (including the secured SSH channels) is in SETUP.md. Building the GPU model machine itself (an Ollama and/or vLLM install reachable under the MM_* variables below) is the machine-side task, and it ships with this repo: the operator-side scripts (setup/, vm_side/), the network fence templates (security/) and the machine-side guides (MACHINE.md, SECURITY.md, SETUP-MACHINE.md) sit next to the server here. Start from MACHINE.md for the machine side.

MCP registration (example with placeholders)

Register the server with your language-model tool. Replace the placeholders in angle brackets with your values. Two ways to point the command at the server, both supported:

From a checkout (no install step, PYTHONPATH points at src/):

{
  "mcpServers": {
    "modelmanager": {
      "command": "/path/to/mcp-modelmanager/.venv/bin/python",
      "args": ["-m", "mcp_modelmanager.server"],
      "cwd": "/path/to/mcp-modelmanager",
      "env": {
        "PYTHONPATH": "src",
        "MM_ACCESS": "direct",
        "MM_VM_HOST": "<your-machine-or-127.0.0.1>",
        "MM_VM_USER": "<your-ssh-user>",
        "MM_CONTAINER_ROOT": "/srv/models"
      }
    }
  }
}

After pip install . (the console entry point, no cwd/PYTHONPATH needed):

{
  "mcpServers": {
    "modelmanager": {
      "command": "/path/to/mcp-modelmanager/.venv/bin/mcp-modelmanager",
      "env": {
        "MM_ACCESS": "direct",
        "MM_VM_HOST": "<your-machine-or-127.0.0.1>",
        "MM_VM_USER": "<your-ssh-user>",
        "MM_CONTAINER_ROOT": "/srv/models"
      }
    }
  }
}

Both blocks follow the plain mcpServers JSON shape most MCP clients read (Claude Code, Claude Desktop and any other MCP client that reads an mcpServers JSON block); consult your client's docs for where this block goes.

MM_ACCESS is direct (the server talks to the service ports under the given address) or tunnel (it talks to 127.0.0.1 and builds an SSH tunnel on demand; only sensible after the isolation, see SETUP.md). The default is tunnel; the example above sets direct because that is what a case a)/b) setup without the level-2 tunnel needs, so set MM_ACCESS=direct explicitly until that tunnel is in place.

Security model: fixed operations

There is no tool that runs an arbitrary command. Every operation is a fixed template in operations.py. The caller can only fill named placeholders, and every value must first pass an allowlist (validation.py): permitted is what explicitly matches the agreed form. It is not filtered out, it is allowed in. On substitution it is additionally quoted with shlex.quote; that is the net below the net, not the safeguard.

Checked on both sides: here before sending and, independently again, by a counterpart running on the model machine, which is bound to the control-channel key. So even if that key is stolen, only the closed set of operations is reachable.

Also: only targets on the own computer or the local network, names are resolved and re-checked at each use (no address pinning like the query server; prefer setting MM_VM_HOST to an IP if you use a hostname). No credentials in the code, no .env. The two SSH keys live outside the repo in your ~/.ssh; the .gitignore additionally excludes key patterns.

How the secured two-channel architecture (data channel for the tunnel, control channel for management) is set up and verified is described step by step in SETUP.md.

Operation tiers and the operator gate

Every tool has a fixed classification, kept in one place in the code (TOOL_TIER) so this table and the policy cannot drift apart:

Tier	Meaning	Tools
READ	only observes, changes nothing	status, models_present, job_status, list_jobs, list_variants, show_variant, list_adapters, training_status, check_training_env, check_isolation
CONTROL	changes running state reversibly	fetch_model, create_variant, switch_service, stop_service, start_service, start_training
DESTRUCTIVE	removes or cancels	remove_model, remove_variant, cancel_training

This classification is not an authorization system (see the note in SECURITY.md); it is a coarse operator switch so you can narrow which operations this server will actually run, without editing code. A disabled operation stays listed over MCP but is refused on call, before any machine access:

Variable	Default	Effect
MM_MAX_TIER	read	Highest tier the server will run. read exposes only READ tools; control exposes READ and CONTROL; destructive exposes all.
MM_ENABLED_OPERATIONS	empty (everything within MM_MAX_TIER)	Comma-separated allowlist of exact tool names, applied on top of MM_MAX_TIER: a tool runs only if it is both within MM_MAX_TIER and, when this list is set, named here. A typo in a name stops the server at startup rather than silently exposing more.

Default is read-only (deny-by-default). Set nothing and only the READ tools run; CONTROL and DESTRUCTIVE operations must be switched on explicitly. The gate refuses a disabled operation early and cleanly, before any SSH or VM access, with a message that names the tier and how to change it. To also allow the management operations (fetch_model, switch_service, training, ...):

"env": { "MM_MAX_TIER": "control" }

Or expose exactly two operations and nothing else, note that MM_ENABLED_OPERATIONS narrows further on top of MM_MAX_TIER, so the tier must already admit both named tools, here switch_service is CONTROL, so MM_MAX_TIER must be raised to at least control:

"env": { "MM_MAX_TIER": "control", "MM_ENABLED_OPERATIONS": "status,switch_service" }

Concurrency guard

The machine has one GPU and one inference service, so two state changes at once would leave it inconsistent. Every GPU-state-changing tool runs its critical section serialized (fetch_model and cancel_training are excepted by design: fetch is a background job, and cancel must always get through): while one runs, a second is refused immediately (not queued), and the GPU-competing operations (switch_service, remove_model, create_variant, start_training, start_service) are refused while a training run this server started is still active. The guard is in-process: it makes this server's own calls consistent. A training run started from a different session is caught on the machine side (a new start_training stops inference first), not fully serialized here; that boundary is stated in SECURITY.md.

The tools

Tool	Purpose
status	What is running: service, model, GPU memory, disk space, containers
models_present	Inventory on the machine, split by Ollama and Hugging Face
fetch_model	Fetch a model, returns immediately with an id
job_status	Progress of a background job
list_jobs	Known jobs here and on the machine
remove_model	Remove, only with a literal confirmation
create_variant	Build a custom Ollama variant: base model, system prompt, parameters
list_variants	Which variants exist, where they come from, what is set
show_variant	What a variant really contains, from both sources
remove_variant	Clear a variant away, only with a literal confirmation
switch_service	Switch vLLM to a different model, with estimate and cross-check
stop_service, start_service	Halt the service and bring it back up
check_training_env	Is the LoRA environment present and importable
start_training	LoRA run in the background, stops inference first and says so
training_status	Step, loss, estimated time remaining
cancel_training	End a run, with confirmation
list_adapters	Finished adapters with their base model
check_isolation	Tunnel works, LAN does not: both by the real behavior

Constraints of a machine with one GPU

• Only one large model at a time. switch_service computes beforehand whether weights plus key-value cache plus base load fit into the allotted memory, and rejects rather than trying. Computed with the real file sizes and the model's config.json, not with a rule of thumb from the name; only if both are missing is there a rough estimate, then explicitly marked as uncertain.
• switch_service may only start a known container image and mount a known path. The container it (re)creates is checked against a closed image allowlist (the configured vLLM image MM_VLLM_IMAGE, default vllm/vllm-openai, plus ollama/ollama, optionally with a tag or digest; a mirrored vLLM image must be set via MM_VLLM_IMAGE or it is rejected) and every bind mount's host side must lie under one of the configured data roots; see SECURITY.md for the full reasoning.
• Inference and training do not share the card. start_training rejects as long as the confirmation to stop inference is not set, and names what would be stopped. Its numeric hyper-parameters (epochs, rank, alpha, learning rate, max length, batch size, accumulation) each have a fixed range that is checked on both sides: here before the run is written, and independently again on the machine before any value reaches the training library.
• The machine does not always have the card. If it is unreachable, the server reports that as a state of its own with an explanation, not as an error. If a second machine sharing the GPU is known via MM_VM2_HOST, the state is gpu_probably_on_second_machine, otherwise simply machine_unreachable.
• A switch that fails silently is the most dangerous failure mode. After every switch it is checked via /health and /v1/models whether the service really answers and serves the desired model. The result is always in the report.

Custom Ollama variants

Ollama lets you derive a model with a fixed system prompt, its own parameters and later a trained adapter, via a Modelfile. The derivation then appears as its own model name and is addressable like any other model. That is the path to specialized agents and, at the same time, the path by which training results enter operation.

create_variant(
    name          = "handyman:v1",
    base_model    = "qwen3:4b",
    system_prompt = "You are a terse expert assistant ...",
    parameters    = {"num_ctx": 8192, "temperature": 0.2})

The delicate spot: free text that runs into a directive file

A system prompt is prose the user must be able to determine. But it lands in a file Ollama reads as a directive list. Passing it through unchecked would bypass the whole protection, not over the shell but a layer above it, in the Modelfile itself. Four measures interlock:

1. The Modelfile is assembled on the machine, not transferred. Only parts go over the control channel: base model, an object of parameter name and number, an adapter path and the system prompt. There is no way to submit a finished Modelfile.
2. The permitted directives are in a closed list. The template can produce FROM, PARAMETER, ADAPTER and SYSTEM and nothing else. FROM takes an Ollama model name and no path and is checked against the actual inventory.
3. The free text travels as Base64. No shell syntax and no newline can arise in transit from the Base64 alphabet; it is unpacked only on the machine, straight into the file.
4. The character set of the free text contains no double quote. A multi-line Modelfile value ends with three of them; with the character missing from the set, the text cannot end its own block and never become a directive. For quotes the typographic quotation marks and the apostrophe are available.

Checked on both sides, since validation.py runs here as on the machine. A crafted request placed straight into the control channel, past the check here, is rejected by the wrapper on its own.

The tunable parameters

Mapped are the ones actually worth using, not everything Ollama knows. Each has a range that is checked before the value reaches the machine.

Parameter	Type	Range	Effect
num_ctx	integer	256 to 131072	how much text the model keeps in view at once
temperature	number	0.0 to 2.0	randomness; 0 means the most repeatable answer
top_p	number	0.0 to 1.0	nucleus sampling by probability mass
top_k	integer	1 to 200	nucleus sampling by count
min_p	number	0.0 to 1.0	lower bound relative to the most probable token
repeat_penalty	number	0.5 to 2.0	repetition brake; 1.0 means off
repeat_last_n	integer	-1 to 4096	lookback of the brake; -1 means num_ctx
num_predict	integer	-2 to 32768	answer length; -1 unlimited, -2 until the context is full
seed	integer	0 to 2147483647	fixed seed for repeatable answers

Adapter: prepared but not live

The adapter parameter is built and checked, but the path has never run, because there is no trained adapter in the reference state. What is present: the allowlist for the adapter path, the conversion to the container's view, the ADAPTER line in the template and the reading in the display tools. What has to be settled with the first real adapter (Safetensors vs. GGUF format, the storage location in the container, the fit to base model and quantization) is noted as open in the code, in place. Until that is proven: leave adapter empty.

Selftest

.venv/bin/python selftest.py          # without the machine, HTTP is mocked
.venv/bin/python selftest.py --live   # additionally against the real machine

Most groups run without a machine; only the optional --live group M needs one. The operator-side setup scripts (setup/, vm_side/) that some checks exercise ship with this repo, next to the server, so with them in place the selftest runs additional checks on this machine, no model machine required: on top of the server-code groups they cover the wrapper in actual execution (group D), the setup script itself (group J), the isolation (group K) and the operator policy, the concurrency guard and the isolation helpers (group N). Allowlists against an attack battery, catalog and substitution, request format, the service side against a stub server, memory estimation against hand-checked values, connection layer, parsing of the machine outputs and the variants all run and are checked. Should you run the server directory on its own, with the setup/ and vm_side/ scripts removed from next to it, the selftest still passes the server-code groups and skips groups D, J, K and the operator-only parts of I and N cleanly with a note, not counted as failed. Neither scenario needs a model machine. Only the --live group M does: it runs against a reachable, set-up machine and is optional on top of either scenario. (The exact check count is what the runner prints on the run in front of you; it grows as checks are added, so it is not quoted here as a fixed number and is not contractual.)

--live does NOT read MM_VM_HOST/MM_VM_USER/MM_CONTAINER_ROOT; it uses its own, separate variables, defaulting to an address that is never reachable (TEST-NET, RFC 5737) so a plain --live run without further setup is silently skipped rather than a false pass:

Name	Default	Meaning
MM_LIVE_HOST	192.0.2.10	Address of the machine group M probes
MM_LIVE_PORT_VLLM	8000	vLLM port on that machine
MM_LIVE_PORT_OLLAMA	11434	Ollama port on that machine

Set all three to point --live at your actual, reachable machine; otherwise you will see "VM not reachable, live group is skipped. Not an error." even though MM_VM_HOST itself is configured correctly for normal server use.

Environment variables

Required fields are marked required; they have no default value.

Name	Default	Meaning
MM_VM_HOST	required	Address of the model machine (127.0.0.1, LAN IP or tunnel/VPN address)
MM_VM_USER	required	SSH user on the machine
MM_ACCESS	tunnel	tunnel (over the SSH tunnel to 127.0.0.1) or direct (straight to the address)
MM_MAX_TIER	read	Highest operation tier the server exposes: read, control or destructive (see Operation tiers)
MM_ENABLED_OPERATIONS	empty	Optional comma-separated allowlist of exact tool names; when set, only those run. A typo aborts startup
MM_VM2_HOST	empty	Optional: second machine sharing the GPU, to interpret unreachability. Leave empty if there is only one machine
MM_KEY_DIR	~/.ssh/modelmanager	Storage of the two SSH keys (outside the repo)
MM_KEY_DATA	${MM_KEY_DIR}/datachannel	Private key of the data channel
MM_KEY_CONTROL	${MM_KEY_DIR}/controlchannel	Private key of the control channel
MM_KNOWN_HOSTS	${MM_KEY_DIR}/known_hosts	Pinned host key file of the model machine
MM_FROM	required (only while building the level-2 allowlist)	Permitted sender address(es) of the two keys, several comma-separated
MM_FROM_NET_ALLOWED	no	Set to yes to let MM_FROM allow a whole network, with a warning (only while building the level-2 allowlist)
MM_REMOTE_PORT_VLLM	8000	vLLM port on the machine (vLLM's standard port)
MM_REMOTE_PORT_OLLAMA	11434	Ollama port on the machine (Ollama's standard port)
MM_LOCAL_PORT_VLLM	8000	Local port under which the tunnel exposes vLLM
MM_LOCAL_PORT_OLLAMA	11434	Local port under which the tunnel exposes Ollama
MM_CONTAINER_VLLM	vllm	Name of the vLLM Docker container
MM_CONTAINER_OLLAMA	ollama	Name of the Ollama Docker container
MM_VLLM_IMAGE	vllm/vllm-openai	vLLM image the recreate allowlist accepts; set this to a mirrored/renamed image, otherwise it is rejected
MM_MIN_DISK_GB	15	Lower bound for an Ollama pull
MM_CONTAINER_ROOT	required, no default	Root directory on the machine under which all model, adapter and cache data live; the server does not start without it
MM_VM_ROOT	${MM_CONTAINER_ROOT}/manager	This tool's own working root on the machine (job folders live beneath it)
MM_VM_DATA_ROOT	${MM_CONTAINER_ROOT}	Root a training dataset path must lie under (data_path_check, df usage report)
MM_VM_ADAPTER	${MM_CONTAINER_ROOT}/adapter	Where a finished LoRA training run stores its adapter
MM_LORA_PYTHON	${MM_CONTAINER_ROOT}/lora-venv/bin/python	Interpreter of the LoRA training environment on the machine
MM_VM_OLLAMA_STORE	${MM_CONTAINER_ROOT}/ollama	Ollama's data folder on the machine (mounted into the container, see table below)
MM_IN_CONTAINER_STORE	/root/.ollama	The same folder's path AS SEEN INSIDE the Ollama container
MM_VM_HF_CACHE	${MM_CONTAINER_ROOT}/hf	Root of the Hugging Face weights cache on the machine
MM_GPU_TOTAL_GB	12.0	Assumed GPU memory in GB, used only when nvidia-smi's real reading fails
MM_AUDIT (machine-side, not set here)	${MM_VM_ROOT}/audit.jsonl	Path of the audit log on the model machine itself; read there by the machine-side wrapper, defaulting to the audit.jsonl that install.sh creates. Not read by this server
MM_ENFORCE_ROOT_OWNED (machine-side, not set here)	unset	Set to 1 to force the machine-side wrapper's root-owned guard on lib/ even in a checkout copy where install.sh has not flipped it on. Can only turn the guard ON, never off, so the environment cannot weaken an installed copy. Used by the self-test; not read by this server
MM_STATE_DIR	$XDG_STATE_HOME/mcp-modelmanager (or ~/.local/state/mcp-modelmanager)	Where this server side stores the local job state (fetch_model/start_training job files)

The local tunnel ports default to the same well-known values as the remote ports; adjust them if those are already taken locally.

MM_CONTAINER_ROOT is required and has no built-in default (there is no default machine layout this tool could assume); without it the server exits at startup. The six MM_VM_*/MM_LORA_PYTHON variables above derive from it by default and stay individually overridable, exactly like MM_IN_CONTAINER_STORE, so a machine with a DIFFERENT directory layout can be adapted through configuration alone, no code change needed. Every value is checked at startup against a strict allowlist for VM paths (absolute, letters/digits/dot/hyphen/ underscore/slash only, no .., no spaces, no shell characters); a malformed value stops the server immediately instead of being trusted. MM_VM_DATA_ROOT and MM_VM_OLLAMA_STORE additionally become the allowlist roots for every path a tool call may pass in later (a training dataset path, an adapter path), so they stay consistent with what is actually configured.

Places on the machine that a variant needs, derived from the roots above (in operations.py, not separately settable). The setup creates these paths:

On the machine	In the container	For
MM_VM_OLLAMA_STORE (default ${MM_CONTAINER_ROOT}/ollama)	MM_IN_CONTAINER_STORE (default /root/.ollama)	Ollama's data folder, mounted
<MM_VM_OLLAMA_STORE>/variants/<name>/Modelfile	<MM_IN_CONTAINER_STORE>/variants/...	the assembled Modelfile
<MM_VM_OLLAMA_STORE>/adapter	<MM_IN_CONTAINER_STORE>/adapter	storage for adapters

License

MIT, see LICENSE.