minion-desktop 1.0.0

Give every AI agent its own computer.

Featherweight Linux desktops (office, dev, research, browser, data, Windows apps) that any agent can use:
Claude, ChatGPT, Gemini, Cursor, VS Code, LangChain, your own code, or a model running on your laptop.
Agents read the screen as text, act in batches or in code, and use any program or website.

English · 简体中文 · 繁體中文 · 한국어 · 日本語 · עברית

Get started

All you need is Docker (or OrbStack or Podman). Then pick one:

1. One click in your AI app:

Claude Code: claude plugin marketplace add apophisss/minion && claude plugin install minion@minion. Any other MCP client: uvx minion-desktop mcp, or npx -y minion-desktop mcp. The buttons, uvx and npx start Minion with uv (npx can use pipx instead); the Claude Desktop extension needs nothing else.

2. One line that installs Minion and connects every AI app on your computer (Claude Desktop, Claude Code, Cursor, VS Code, Windsurf/Devin, Gemini CLI, Codex, LM Studio, Cline, Kiro):

curl -fsSL https://raw.githubusercontent.com/apophisss/minion/main/install.sh | sh        # macOS, Linux

irm https://raw.githubusercontent.com/apophisss/minion/main/install.ps1 | iex             # Windows

It ends with minion setup, which checks Docker, prepares a desktop and runs minion connect. Run either one again any time.

3. From ChatGPT, Claude.ai or Le Chat in the browser: run minion share and paste the URL it prints into the app's connector settings (how).

Then just ask: "Create an office desktop, write a one-page brief about our Q3 numbers in Writer and save it as brief.docx." Every way to install is in docs/install.md.

_{▶ Watch the one-minute demo: an agent researches a trip on Wikipedia, makes a budget with a chart and writes the plan as a PDF. 12 tool calls and 2,372 tokens of text; the same steps with screenshots would cost about 36,855. Recorded live with python scripts/demo_video.py, which makes the calls an agent would make.}

Why Minion

• 🖥️ Any program, any website. Every desktop is a real Linux computer: browsers, LibreOffice, IDEs, terminals, file dialogs and installers, plus Windows programs through the windows (Wine) profile. Open any app by name and it is installed the first time (open gimp, open vscode, open dbeaver, any Debian package), web apps open in the browser (open gmail), and the first open answers with a few tips for the app. 130+ apps by name.
• 🧭 Shortcuts through every app. see --menus lists every menu command with its keyboard shortcut, and menu "File > Export as PDF" runs one without opening a menu. path ~/out.pdf fills any Open/Save dialog. Right-click menus show up in see, button:Save picks one kind of control, and a button below the fold is scrolled into view. Any document reads as text (read report.docx, xlsx with formulas, pptx, pdf) and converts in one call (convert report.docx pdf).
• 🧠 Agents read the screen as text. see turns any window into a list of controls with stable @ids, taken from the accessibility tree with OCR as a fallback: @12 button "Save" (210,62). Web pages, editors (with line numbers), terminals, dropdowns with their options, and spreadsheets (row by row: row 2: A2 North · B2 120) all come out as exact text. When pixels matter, screenshots can draw the @ids for you.
• 🎯 Honest actions. Every action waits for the screen to settle (for web pages to load, and while an app says "Loading…") and reports what happened: windows, a page's alert or prompt, files saved or downloaded, a pop-up that took the click. Text targets must really be on screen: a miss fails with the closest candidates instead of clicking something else. Tested on 93 everyday tasks done by AI agents.
• 🪙 Built to save tokens. You get diffs instead of re-reads (see(changes=true): about 60 tokens instead of about 900). Code mode runs a 50-step GUI loop in one call, crops and half-size screenshots cut image cost, a compact mode shrinks the tool definitions from about 2,100 tokens to 230, and the Agent Skill costs about 100 tokens until it is used.
• 🪶 Featherweight. One small container per desktop (Xvfb, a 2-process window manager and taskbar, a small agent), with no VNC and no desktop environment. Desktops are ready in about 1.5 s and use about 60 MB when idle. Idle desktops are frozen automatically (0% CPU) and wake in about 0.2 s.
• 🔌 Works with every agent. One toolset everywhere: MCP (local or remote), function calling for OpenAI, Anthropic, Gemini and local models, HTTP with OpenAPI, a CLI and an Agent Skill for shell agents, Python and TypeScript SDKs, and native adapters for Claude, OpenAI and Gemini computer use.
• ⚡ Parallel by design. create count=8 starts a desktop per worker, fork clones one before a risky step, and sessions keep agents apart.
• 🙋 Humans in the loop. A live view in the browser offers one-click take control. Agents can hand off ("please sign in") and wait for Done.
• 🎞️ Teach once, replay forever. Every action is recorded. Export an HTML timeline or GIF, or turn the session into a Python script that replays it with no model at all.
• 📦 Zero dependencies. The host side is pure Python standard library. Desktops run in Docker or Podman (macOS, Windows, Linux), or natively on Linux.

A quick tour from the shell

Any agent with a terminal (Claude Code, Codex CLI, Aider, Open Interpreter, Goose) can use the CLI directly, and minion skill --install teaches it how:

minion up research
minion open en.wikipedia.org             # waits until the page has loaded
minion fill "Search Wikipedia" "Ada Lovelace" && minion key Return
minion wait-for "Analytical Engine"      # says where it saw the text
minion see --full                        # read the article as text
minion select @14 "Title"                # pick a dropdown option
minion shot -o page.png --marks          # a screenshot with @ids drawn on it
minion read ~/Downloads/paper.pdf         # any document as text: pdf, docx, xlsx, pptx, epub...
minion open gimp                          # any app by name: installed the first time
minion menu "File > Export As > Export as PDF..."   # any menu command
minion path ~/Documents/report.pdf        # fill the open Save/Open dialog
minion cp :~/Documents/notes.md .        # copy files out (and in)

_{Menus, file dialogs and apps by name: menu "File > Export As > Export Directly as PDF", path ~/Documents/report.pdf, then open gimp, which installed GIMP in 16 s. Recorded by the flight recorder.}

Works with every agent

A complete agent with any OpenAI-compatible model ("anthropic", "gemini" and "openai-responses" work the same way):

from openai import OpenAI
from minion.tools import Toolkit

tk = Toolkit()
client = OpenAI()  # or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") for a local model
messages = [{"role": "user", "content": "On a research desktop, find the population of Iceland on Wikipedia."}]
while True:
    msg = (
        client.chat.completions.create(model="YOUR-MODEL", messages=messages, tools=tk.definitions("openai"))
        .choices[0]
        .message
    )
    messages.append(msg)
    if not msg.tool_calls:
        break
    messages += tk.run_calls(msg.tool_calls)  # runs every call, returns the messages to send back
print(msg.content)

Python, and code mode

from minion import Desktop

with Desktop.create("office", keep=False) as desk:  # removed when the block ends
    desk.open("calc")
    result = desk.script("""
for item, price in [("Coffee", 3.5), ("Bagel", 2.25), ("Juice", 4)]:
    write(f"{item}\\t{price}\\n")             # Tab moves right, Return goes down
write("Total\\t=SUM(B1:B3)\\n")
print(cells()["B4"])                          # visible cells as {"A1": ...}
""")
    print(result.stdout)  # 9.75
    desk.key("ctrl+s")
    desk.fill("Name", "q3.xlsx")
    desk.key("Return")
    desk.wait_for("q3.xlsx")
    desk.download("~/Documents/q3.xlsx", ".")

script runs Python inside the desktop with click · write · key · scroll · fill · see · elements · cells · wait_for · launch · run. Loops cost one call, and only what the code prints comes back. The Desktop API also has screenshot · see · act · open · run · upload · download · clipboard · windows · freeze · fork · snapshot · restore · recording · create_many. See docs/python.md.

Native computer use (Claude, OpenAI, Gemini)

Minion is a ready environment for each vendor's computer-use model, with screenshot scaling and coordinate mapping handled for you:

import anthropic
from minion import Desktop
from minion.integrations.anthropic import MinionComputer, allow_all

desk = Desktop.create("office")
with MinionComputer(desk, confirm=allow_all) as computer:  # the desktop is the sandbox
    runner = anthropic.Anthropic().beta.messages.tool_runner(
        model="claude-opus-5-5",
        max_tokens=16000,
        tools=[computer],
        messages=[{"role": "user", "content": "Make a 3-slide deck about otters in Impress."}],
    )
    for message in runner:
        print(message)

Claude's computer_toolset_20260801 (all 17 actions) and the earlier computer_20251124 are covered in docs/computer-use.md. OpenAI and Gemini computer use are covered in docs/computer-use-openai-gemini.md.

Profiles

Your own profile is a 5-line TOML file in ~/.config/minion/profiles/. It is built automatically the first time you use it:

# ~/.config/minion/profiles/designer.toml
extends = "browser"
description = "Firefox plus GIMP and Inkscape."
packages = ["gimp", "inkscape"]
apps = [{ name = "GIMP", command = "gimp" }, { name = "Inkscape", command = "inkscape" }]

Or add software at runtime: minion run "minion-install gimp https://example.com/tool.AppImage pip:yt-dlp". See docs/profiles.md.

How agents see the screen

see is the agent's main sense. Here it is on a LibreOffice window:

Screen 1280x800 · 1 window
  [1] "Untitled 1 — LibreOffice Writer" — libreoffice-writer (focused) at 0,0 1280x770

UI of "Untitled 1 — LibreOffice Writer" (click/fill with @id; (x,y) is the element center):
@1 menu "File" (22,38)
@2 menu "Edit" (62,38)
…
@12 document text "Untitled 1 - LibreOffice Document" [editable] (603,424)
  @13 heading "Q3 Market Brief" (603,247)
  @14 paragraph "Prepared by an AI agent on a Minion desktop. No human touched the keyboard." (603,282)

After an action, observe="changes" (or see(changes=true)) returns only the difference:

✓ typed 30 chars
→ screen changed <1% (region 404,226 236x20)
Changes since your last look 1.0s ago (81 elements unchanged; their @ids still work):
  + @84 paragraph "Quarterly numbers look strong." [focused] (603,234)
  ~ @80 label "4 words, 30 characters" (281,730)

Text that accessibility doesn't expose (a slideshow, a canvas app, a picture of text, a remote screen, a program without an accessibility tree) is read with OCR, only where it is missing and in the screen's own language: the scripts in the window's titles and labels choose it. Hebrew, Arabic, Russian, Chinese, Japanese, Korean and Hindi are built in, and other languages install on first use. Large text and light text on colour get passes of their own, and a picture that hasn't changed is never read twice. On 24 photographed Hebrew slides, see reads 94% of the words. Here it is on a web page that shows a slide as an image:

@2 heading "Board meeting, October 15" (640,131)
@3 paragraph "The slides from the quarterly review:" (640,186)

Text in the picture at 0,117 800x480 (OCR: accessibility doesn't have it):
  @4 "Q3 review" (146,254)
  @5 "Revenue grew 12% to $4.1M" (218,348)
  @6 "1,240 new customers, churn down to 2.1%" (291,404)
  @7 "Next: launch in Japan and Brazil" (239,460)
  @8 "| Goal: $5M in Q4" (163,555)

When pixels matter, screenshot --marks labels every control with its @id:

Efficiency, measured

Five real tasks (a PDF report, a spreadsheet, a Wikipedia lookup, a web form, reading a document) cost an agent 4,356 tokens with Minion and 50,505 with screenshots: 11.6× fewer. The full table and method are in docs/benchmarks.md; reproduce it with python scripts/bench_tasks.py.

These were measured on a 2-vCPU VM. Reproduce them with python scripts/bench.py.

Watch, take over, hand off

minion view (or the view tool) opens a live view of any desktop in your browser. Click Take control to use your own mouse and keyboard. An agent can call view(message="Please sign in to your Google account"): you get a banner, sign in, press Done, and the agent continues. A good pattern is to log in once, snapshot, then fork the logged-in desktop for each task. minion serve adds a dashboard with live thumbnails:

How it works

flowchart LR
  subgraph host["Your machine"]
    A["Any agent<br/>(MCP, function calling, HTTP, CLI, SDK)"] --> T["minion tools<br/>(pure-stdlib Python)"]
    T --> V["Live view and dashboard<br/>(minion serve)"]
  end
  subgraph desk["Desktop (one container each)"]
    S["minion agent<br/>RPC over docker exec"] --> X["Xvfb + JWM"]
    S --> AT["AT-SPI<br/>accessibility tree"]
    S --> O["Tesseract OCR"]
    S --> C["code mode<br/>(Python inside)"]
    X --> Apps["Firefox · LibreOffice · Terminal · Wine · …"]
  end
  T -->|"docker exec (no open ports)"| S

Read docs/design.md for the reasoning behind each choice, and docs/architecture.md for the internals.

Security

Each desktop is an unprivileged container with no host mounts by default. Use --offline to cut network access, or limit resources with --memory and --cpus. The HTTP server binds to 127.0.0.1, needs a random token, and refuses cross-site and DNS-rebinding requests. File transfers to and from your computer stay inside one folder, and minion share URLs carry a fresh token and stop working when you stop sharing. The desktop user has passwordless sudo inside the container, so agents can install software. Treat everything on a screen as untrusted input (prompt injection), and keep credentials out of desktops unless a human hands off. Read SECURITY.md.

Documentation

Website · Install · Connect your apps · Remote MCP · Design · Agents & MCP · Integrations · CLI · Python · HTTP API · Claude computer use · OpenAI & Gemini computer use · Profiles · Apps by name · Windows apps · Architecture · FAQ

Contributing

Questions and ideas are welcome in Discussions, and bugs in issues. To help build Minion, pick a good first issue (translations, apps, agent apps, benchmark tasks), read CONTRIBUTING.md, and see where it's going in the roadmap.

License

MIT. The desktops contain open-source software under their own licenses (Debian, Firefox, LibreOffice, Wine, …).