hobkin

An AI agent for austere environments: one stdlib-only Python file, no installs, no open ports. Use it from the command line, or as a worker on hearthboard. (A hobkin is a little hob, the house spirit that does the chores while the house sleeps.)

Get one running

Anywhere with Python 3.9+ (Linux or another POSIX system). Nothing to install:

curl -O https://raw.githubusercontent.com/frank-lynam/hobkin/main/hobkin.py
python3 hobkin.py https://openrouter.ai/api/v1 anthropic/claude-sonnet-4.5 sk-or-... --chat

As a background service: leave out --chat, and name it if you like. It runs as a systemd user service where there is one (in the background where there isn't), and keeps running after you log out. Talk to it whenever:

python3 hobkin.py https://openrouter.ai/api/v1 anthropic/claude-sonnet-4.5 sk-or-... --id helper
python3 hobkin.py chat helper

As a hearthboard worker: add the board's URL and the agent's token. The board's Users page shows this line, ready to copy, when you add an agent:

python3 hobkin.py https://openrouter.ai/api/v1 anthropic/claude-sonnet-4.5 sk-or-... \
  --hearthboard http://board.lan:43917 at_THE_AGENTS_TOKEN

More workers on the same machine reuse the model: python3 hobkin.py --hearthboard http://board.lan:43917 at_ANOTHER_TOKEN.

Installed rather than copied: pipx install hobkin (or uv tool install hobkin), then the same commands as hobkin ....

What that does

That:

• checks the key works and the API offers that model (a typo gets the close names back; --force skips the check for APIs that don't list their models);
• sets it up: on a fresh install, as the default model in ~/.hobkins/_shared/config.json, with the key in _shared/keys/, readable only by you. On a machine that's already set up it changes nothing shared: the model and its key go in the new agent's own folder, and your other agents and settings stay as they are. Leave the key out and it asks for it, so it stays out of your shell history; leave it out for a local server with none;
• starts a new agent as a systemd user service (see "Keeping agents running"), or in the background where there's no systemd (or with --no-service);
• with --chat, connects you to it. Without, it prints how.

The API style comes from the URL (anthropic.com is Anthropic's; anything else is OpenAI-style: OpenRouter, vLLM, llama.cpp, Ollama's /v1, LiteLLM...); --api overrides it. --id NAME names the agent instead of a made-up name like bodkin. Run it again with another URL or model to start another agent on that one.

To see what an API offers before choosing: hobkin models URL [KEY]; hobkin models lists your configured model's API, and /models does the same in a chat.

With --hearthboard URL TOKEN, it also checks the token with the board, then:

• saves the board's own skill (/api/skill) in that agent's own skills/, so it works by the board's current rules (and agents here that aren't workers don't mistake the board for their work system);
• puts the token in that agent's secrets.env (readable only by you; the model only ever sees it as an environment variable, never in its context);
• sets its idle prompt to the board's wait loop, so it picks up work itself;
• names the agent after its board user, unless you give --id.

Without URL MODEL KEY it uses the model you've already set up.

Why it's built this way

• One file, standard library only. hobkin.py and Python 3.9+. Nothing to install, so it runs on locked-down servers, containers and old boxes alike.
• No ports. It listens on nothing; it only makes outgoing calls to the model API (and your board, if it has one).
• Files are the interface. Talk to it with echo, watch it with tail -F, and see exactly what the model sees with cat.
• Many per account. Each agent has its own id and folder, and a lock stops two processes running the same one.
• With or without a work system. Give it skills for a tracker or a queue and it pulls work from there; with none, its work is what you ask it in the chat, and between requests it waits rather than wandering the machine.

The long way

hobkin                 # a new agent: prints its name, e.g. bodkin
hobkin bodkin          # resume that agent later (its context is kept)

The first run creates ~/.hobkins/_shared/config.json. Set your model there (see Models) and make sure the API key is reachable.

Then, from another terminal, chat with it:

hobkin chat bodkin

In a terminal it's in colour, so things are easy to find when you scroll back: the agent's name in magenta, file names, paths and code in green, bold in bold, links underlined, tool calls dimmed and failed commands in red (agents are asked to put names of things in backticks for this). Piped, or with NO_COLOR set, it's plain text. Its replies appear as they come, and each tool call as one line (a command, a file read, a failed exit status), without its output; hobkin chat ID --full shows the output too, as the out file has it. Each line you type (or paste) is sent; /stop and the other commands work, and Ctrl-D leaves. Output never wipes out what you're typing (the line is redrawn below it), and you get the usual line editing: ↑/↓ for earlier messages (kept in the agent's .chat_history), ←/→, Ctrl-R to search. Piped, it sends the input, prints the reply and exits: echo "status?" | hobkin chat bodkin.

Or with plain files:

echo "Clone https://example.com/repo and get its tests passing" > ~/.hobkins/bodkin/in
tail -F ~/.hobkins/bodkin/out

Run it in the background however you like: nohup hobkin bodkin &, tmux, or a systemd user service.

An agent's folder

~/.hobkins/ID/ (or $HOBKIN_HOME/ID/):

~/.hobkins/_shared/ holds what all agents share: config.json, AGENTS.md (how to work as a hobkin: a task routine with checklists, failure handling, organization, memory and tidying up; created on first run, edit freely: it's never overwritten, so new defaults reach it only if you delete it), an optional SOUL.md, any other .md instructions, and secrets.env.

The system prompt is:

1. SOUL.md, then AGENTS.md, then the other .md files in name order;
2. the list of skills (see below);
3. MEMORY.md.

Each .md file is taken from the agent's folder if it has one, otherwise from _shared/. The prompt is rebuilt every turn, so edits take effect on the next one.

Skills

Skills use the same format as Claude Code and other agents: a folder holding a SKILL.md with a name and description in its front matter, plus any scripts or reference files. Install one by copying its folder into ~/.hobkins/_shared/skills/, and every agent can use it.

~/.hobkins/_shared/skills/release/SKILL.md
~/.hobkins/_shared/skills/release/build.sh

The prompt lists only each skill's name and description. When a task calls for one, the agent opens it with the skill tool. That gives it the instructions, the skill's folder as a full path, and its list of files, so it never has to work out where scripts live. Dozens of skills cost a line each.

• An agent's own skills/ folder can add skills, or replace a shared one with the same name.
• skill_dirs in config adds more folders at the lowest priority, e.g. ["~/.claude/skills"] to reuse skills you already have.
• New skills are picked up on the next turn; no restart needed.

Talking to it

Anything you write to in is a message, and the agent is told plainly what to do with it, the way a work tracker's API would:

• When it's free: [message from frank, Sat 2026-10-17 09:12: they're talking with you now. Answer them in your reply]. Set "operator": "frank" in config.json (the name people know you by, e.g. your tracker username) and its system prompt says who that is; without it, [message from operator, …].
• When it's busy: the message rides along with the result of the command it was running: [message from frank, …, while you were working: answer it in your next reply, then carry on]. If it goes on working without a word, it gets one reminder that you're waiting for an answer.
• While you're talking with it, the idle prompt holds off for 5 minutes after each message, so it doesn't wander off between your messages.
• After a quiet spell the label says how long it's been (; 3d 2h since you last worked), so Friday's request isn't mistaken for today's.

Automatic messages are marked and timed too ([from hobkin: idle prompt, …], [Reminder #3 …], finished jobs), so the agent never mistakes them for you. A line that is just one of these is a command instead:

hobkin chat ID puts both in one terminal (see Quick start). hobkin list shows all agents, their status and their last line of output.

Models

config.json names the model to use and describes each one:

{
  "model": "claude",
  "models": {
    "claude":     {"api": "anthropic", "base_url": "https://api.anthropic.com",
                   "model": "claude-sonnet-5-5", "key_env": "ANTHROPIC_API_KEY", "key_file": "~/.anthropic.key"},
    "openrouter": {"api": "openai", "base_url": "https://openrouter.ai/api/v1",
                   "model": "anthropic/claude-sonnet-5.5", "key_file": "~/openrouter.key"},
    "local":      {"api": "openai", "base_url": "http://127.0.0.1:8080/v1", "model": "local-model"}
  }
}

model picks one of models. The models are exactly the ones in your config files (the three above are only the starter file), so a model you delete is gone. An agent's own config.json can add models, or change settings on a shared one (e.g. its requests_per_minute), and can pick a different model. If model names one that isn't defined, hobkin stops and lists the ones that are; if it's left out and there's only one model, that one is used.

• api: anthropic (the Messages API) or openai (Chat Completions: OpenAI, OpenRouter, vLLM, llama.cpp, Ollama, ...).
• The key comes from the environment variable key_env, or else from the file key_file. It's removed from the agent's shell environment.
• headers (optional): extra HTTP headers.
• thinking (optional): "adaptive", a token budget (8000), or the API's own object. Anthropic only; it must be smaller than max_output_tokens.
• effort (optional): "low" to "max". It goes to Anthropic as output_config.effort, to OpenRouter as reasoning.effort, and to other OpenAI-style APIs as reasoning_effort.
• extra_body (optional): anything else your provider takes, merged into every request as is.
• prompt_cache (optional): on OpenAI-style APIs, mark the system prompt and the newest message for caching. It's on by default for Claude models (e.g. on OpenRouter), where re-reading the conversation then costs about 5% of the normal input price; a test session went from about $0.023 per call to $0.001. On OpenRouter, each call's reported cost is added up in state.json (spend_usd).
• requests_per_minute (optional): a cap on calls to this model. It's shared by every agent using the same API, model and key, because that's what the provider counts. The agents coordinate through a small lock file in _shared/ (holding a hash of the key, never the key). An agent at the cap waits, and still answers commands such as /stop while it does.

When the provider rate-limits anyway (HTTP 429):

• It waits exactly as long as the API says. It reads Retry-After (seconds or a date), OpenRouter's X-RateLimit-Reset, Anthropic's anthropic-ratelimit-*-reset, and reset times inside the error body, then waits until then plus a second.
• If the API gives no reset time, it waits 15 seconds, then 30, 60…, up to 5 minutes. A 40-second limit costs two blocked calls at most.
• The cooldown is shared. The first agent to hit a 429 records "blocked until …" in _shared/, and every other agent on the same API, model and key waits it out without trying, so a team causes one blocked call, not one each.
• It's visible:
  • the log says rate limited by the API; the API said when (retry-after): waiting 41s (until 13:42:10);
  • state.json says waiting (rate limited);
  • the heartbeat says waiting out a rate limit (until 13:42:10). /stop and the other commands still work while it waits.

The context window is asked for, not hard-coded. At start-up hobkin asks the API:

• Anthropic: GET /v1/models/{model} (max_input_tokens, max_tokens).
• OpenAI-style: the /models list (context_length as on OpenRouter, max_model_len as on vLLM, and similar).
• llama.cpp's /props and Ollama's /api/show.

context_tokens on a model is optional: set it and that number is used as given; leave it out (or null) and it's detected. If the API doesn't say, set it. Failing that, hobkin learns the limit from the first "context too long" error and remembers it. Windows can change (a local server restarted with a different size): the API is asked again before each compaction, after a "too long" error that doesn't say the limit, and on /limits. state.json records where the number came from.

When the conversation passes compact_at (80%) of the window, the agent writes itself a handoff note. The conversation is replaced by that note, with the newest unanswered message carried over unchanged, and the old conversation goes to archive/ (messages only: the system prompt is rebuilt every turn anyway), along with out, which starts afresh. The note tells the agent where the archive is, so it can grep it.

Fallback models: "fallback_models": ["other", "local"] (names from models) are tried in order when the model refuses outright: withdrawn, renamed, a bad key, out of credit. Free models on OpenRouter come and go, so it's worth having one. Busy and rate-limited models are waited out instead. A restart goes back to model.

Typos and wrong types in config.json are reported at startup (in out and on stderr), with suggestions such as "unknown setting 'idel_seconds' (did you mean 'idle_seconds'?)". They're warnings; nothing is refused.

Other settings (shared or per agent): bash_timeout (600 s per command by default; the agent can ask for any longer timeout on a command), output_limit (30,000 characters of command output, head and tail kept), max_output_tokens (16,000, capped at what the API reports), and idle_prompt / idle_seconds (see below).

Its tools

• bash runs as your user, starting in work/. It behaves like one shell that stays open:
  • The first command runs as a login shell (your profile's PATH and so on).
  • After every command, its state is saved to shell.sh: the folder, variables, exports, functions and aliases. The next command starts from that state. So cd, export and source .venv/bin/activate carry over, even across restarts.
  • Each command is still its own process, so a hung or crashed command can't take the shell's state with it. Shell options like set -e don't carry over.
  • reset: true starts a fresh login shell in work/.
  • Every result ends with the folder it's now in.
  • The environment has secrets.env plus HOBKIN_ID, HOBKIN_DIR, HOBKIN_WORK and HOBKIN_MEMORY.
• read_file shows a file with line numbers (in pages), or lists a folder.
• write_file writes a whole file. edit_file replaces one exact piece of text. The text goes in as is, with no shell quoting, which is where smaller models most often go wrong.
  • Errors are written to help: "appears 2 times (lines 3, 5)", or the closest matching lines when the text isn't found.
  • Line numbers pasted from read_file are stripped automatically, and Windows line endings are handled.
  • Paths are relative to the shell's folder; $HOBKIN_WORK, $HOBKIN_MEMORY and ~ work.
• remind sets a reminder to wake itself later, and reminders lists or cancels them (see below).
• skill opens a skill by name.
• view_image shows it an image (PNG, JPEG, GIF, WebP, under 5 MB): a screenshot of the UI it built, a chart, a photo. On OpenAI-style APIs the image follows the tool result as a user message, because tool messages can't hold images.
• fetch_url reads a web page as text: headings, lists and tables kept, scripts and menus' styling dropped, links numbered with a list at the end. Long pages come in parts (offset, limit), JSON is pretty-printed, and save_as downloads anything else (a PDF, an image) into the workspace.
• web_search searches DuckDuckGo and returns the top ten results: title, URL, snippet.

The web

Normal sites need no setup. For an internal wiki behind a client certificate (PEM files), set it in config:

"web": {
  "client_cert": {"cert": "~/certs/me.pem", "key": "~/certs/me.key",
                  "ca": "~/certs/corp-ca.pem", "password_file": "~/certs/key-password"}
}

• One certificate for everything: it's offered to any site that asks for one, including SSO servers along a redirect chain. Most public sites never ask. ca (your internal CA, trusted alongside the usual ones) and password_file are optional.
• SSO logins that need no typing are followed: redirects, meta refreshes, and the auto-submitting forms that SAML and OpenID logins use (up to 8 steps). Cookies are kept in the agent's cookies.txt, so the next page skips the login. A login that wants a password shows up as the login page's text.
• Pages built by JavaScript come back nearly empty. Give the agent a skill for those (an API, or a headless browser).
• Search uses DuckDuckGo's HTML page (web.search_url, no key). DuckDuckGo sometimes answers servers and VPNs with a "bots use DuckDuckGo too" check instead of results; the agent is told so, and searching from a home connection usually works. To check a machine: curl -s -A Mozilla/5.0 -d q=test https://html.duckduckgo.com/html/ | grep -c result__a (a number above 0 means results).

Background jobs

bash with background: true returns at once ("background job #3 started"), and when the job ends the agent gets a message with its exit code, how long it took, and the last of its output. Jobs run detached in the shell's current state, with their output in jobs/N/out.log. They outlive a hobkin restart: a job that finished while hobkin was stopped is reported when it starts again. That suits builds, test suites, downloads, and "naps", where a job switches off a local model, uses the GPU, and starts the model again: nothing calls the model while the job runs, and its end wakes the agent.

Announce first

Commands whose effects leave the machine are announced before they run:

• What counts (announce_first in config, a name and a regex each): force pushes, pushes to main, publishing packages, releases, image pushes, infrastructure changes, cloud deletions, and sending email or webhook messages.
• The first try isn't run. hobkin tells the agent to post what it's about to do and why where people will see it, then run the same command again with announced set.
• Then it waits announce_delay (2 minutes), long enough for someone to object in chat. After that, the same command runs.
• To stop it, send /veto, which blocks everything waiting. Or just object where it posted: the agent is told to check before it runs the command.

Everything else, anything inside the box, needs no announcement.

Daily budget

daily_budget_usd pauses an agent for the rest of the day once it has spent that much. First it gets one last turn to say so where people will see it. Spend counts only where the API reports each call's cost (OpenRouter does); state.json keeps spend_today and the all-time spend_usd.

Nothing asks for approval: an agent can do anything your account can. Agents in one account stay apart by convention (their own folders), not by enforcement. Use a separate account or container when that matters.

Safe mode

For a system people depend on: "safe_mode": true in an agent's config.json (or --safe on the one-line setup). The agent can read what its account can read, but its commands can only write in its workspace, its memory files and its job logs, with a private temp folder instead of /tmp.

• It's enforced by the kernel, not by hobkin checking commands. Each command runs through a small helper that locks itself down with Landlock and then runs bash. Everything that command starts is locked down too, and it can't be undone from inside. sudo won't work either, because nothing locked down can gain privileges. hobkin's own file tools follow the same rule, and the agent can't change its own config.json to turn safe mode off.

• What's covered depends on the kernel, checked at start-up and written to the log, to state.json and into the agent's instructions:

  Linux	Landlock	Blocked
  5.13+	v1	writes outside the workspace
  6.7+	v4	...and TCP connections ({"safe_mode": {"network": true}} allows them)
  6.12+	v6	...and signals to other processes

  With no Landlock at all (an older kernel, it's turned off, or a container's security profile blocks it), a safe-mode agent won't start. It never runs unprotected.

• Not covered: reading. Whatever it reads goes to the model provider, so run it as an account that can read only what it should. On kernels below 6.12, also run it as a separate user, so it can't signal your services' processes. Local services reached through socket files (Postgres, Docker) aren't blocked by Landlock either; a separate user without access to them covers that.

• How it works with people: it investigates by reading. When something needs changing, or a command it can't run, it writes a script in work/for-review/ (what it does, why, and for changes, how to undo them) and asks you to review and run it. The script saves all its output to work/for-review/logs/, and gathers plenty while it's there (versions, config, the state before and after), so the agent reads the results itself: you run it and say "done".

Reminders

An agent can schedule its own wake-ups: remind with in ("20m", "2h", "1h30m"), or at (local time: "2026-10-17 09:00", "09:00" for the next one, "tomorrow 09:00"), optionally every to repeat. The reply states the exact time it will fire, because small models are bad at time arithmetic. When it's due, the agent gets a message: "[Reminder #3, set … for …. It's now …]" and then its own text.

• Idle: it wakes straight away.
• Mid-task: nothing is interrupted. The reminder arrives with the result of the step in progress (after a long command finishes, for example), like your own mid-task messages.
• Paused: it waits until /resume.
• While hobkin wasn't running: it's delivered at the next start, marked late. A repeating one is delivered once, saying how many times were missed, and moves to its next time.
• No limits. Any number of reminders, repeating as often as it likes, even every second. A repeating reminder that has fallen behind is delivered once, with a count of how many times it was missed, so it can't flood the agent. Using every where in was meant is the classic slip, so the reply points that out.
• reminders.json is a plain file: read it, edit it, or delete entries. The agent's current reminders are also listed in its system prompt.

Keeping agents running

Reminders and idle prompts only fire while the agent's process runs. An idle hobkin costs nothing (it sleeps; no API calls), so keep agents up as systemd user services:

hobkin service add tickets-1      # start it now, and at every boot
hobkin service remove tickets-1   # stop it and take it off boot

add does all of this:

• It writes one template unit, ~/.config/systemd/user/hobkin@.service, shared by all agents.
• It points that unit at the hobkin you ran it with (the installed command, or python3 /path/hobkin.py).
• It enables and starts hobkin@tickets-1.
• It turns on linger, so agents keep running when you're logged out. If your system won't allow that without a password, it tells you the command to run.

hobkin list shows which agents run as services.

• Stopping and crashes: /stop exits cleanly and stays stopped. A crash is restarted after 30 seconds, and systemd gives up after 5 failures in 10 minutes, so a broken config can't spin.
• Logs: the agent's own log stays in its out file. The journal (journalctl --user -u hobkin@ID) gets startup errors, plus a heartbeat every heartbeat_seconds (60) while the agent is on duty:
  • hobkin ID active: working while it works;
  • hobkin ID active: on duty, next check in 7m (reminder #2) while it's idle with a wake-up scheduled (say, a tracker check with nothing to pick up).
  • It's silent while only a message can wake the agent, or when it's paused, so a quiet journal means it's waiting for you. There's no heartbeat in a terminal; set heartbeat_seconds to 0 to turn it off.
• Alerts: when an agent pauses on its own (a model error, the daily budget) or falls back to another model, it may not be able to say so itself. Set alert_command to a shell command and it runs with the news in $HOBKIN_ALERT and the agent's secrets.env in its environment, e.g. curl -s -H "Authorization: Bearer $TICKETS_TOKEN" --data-urlencode "text=$HOBKIN_ALERT" https://tickets.example/api/comment.
• Keys: a service doesn't see your shell's environment. key_file works as is; for key_env, put NAME=value lines in ~/.config/hobkin/env (chmod 600). add warns when that's needed.
• Another HOBKIN_HOME gets its own template (hobkin-<hash>@.service), so homes never clash.

Waking up when idle

Hobkins don't nap. After idle_seconds (60) of quiet, an idle agent gets idle_prompt, labelled [from hobkin: idle prompt]. The default is "You're idle. Unacceptable! Grab work from your work sources, or hunt for something useful to do." It stays quiet while the agent has a reminder set or a background job running. Set idle_prompt to null to let it rest until a message or reminder arrives. Each nudge is a model call, so pair a short idle_seconds with a requests_per_minute you're happy with.

The gremlin

The default AGENTS.md makes a hobkin a relentlessly cheerful, work-greedy, cheeky little gremlin:

• Forgiveness over permission. It acts, then says what it did.
• Good enough beats better. It ships the working version and moves on.
• The opposite of a yes-man. It laughs at bad ideas, its own included, and does the better thing.
• Hot takes. It rates each task out of five stars.
• Never fakes success.

Small models skim long instructions, so hobkin also nudges at the moment each behaviour matters:

• A message from you carries a reminder to give a hot take first (a hello or a question just gets answered).
• A clear slip (a file-tool error, a command that wasn't found) gets a nudge towards the blooper reel.
• Writing that loses its voice (a reply, or a .md/.txt file, heavy with committee words like "robust", "comprehensive", "production-ready") gets "that reads like a committee wrote it", once per turn.
• A file read only partly ends with how many lines are still unread, so the agent doesn't fill the rest in from imagination.
• When a working turn ends, there's one wrap-up check: did you report where the people are, any bloopers, more work to grab? It's sent once per turn, so it can't loop.

Busy or self-managing agents get no chatter. If the agent has set up its own next wake-up (a reminder, or a background job still running), the hot take, the wrap-up check and the idle nag stay quiet; it's clearly managing itself. A message that arrives mid-task gets no hot-take reminder either. The corrective nudges (echo, slips, committee voice, unread lines) still apply, since they cost nothing extra.

All of it is plain text in hobkin.py and AGENTS.md, so rein it in as you like.

As a hearthboard worker

Hearthboard (formerly agent-tickets) is a ticket tracker for agents. --hearthboard URL TOKEN (see the top) sets an agent up for it in one go; the running agent knows nothing about hearthboard beyond the board's skill. By hand, that's:

hobkin                            # note the id, then /stop it (Ctrl-C)
ID=bodkin URL=http://127.0.0.1:43917 TOKEN=the-token-for-this-agent
printf 'AGENT_TICKETS_URL=%s\nAGENT_TICKETS_TOKEN=%s\n' "$URL" "$TOKEN" > ~/.hobkins/$ID/secrets.env
chmod 600 ~/.hobkins/$ID/secrets.env
curl -s -H "Authorization: Bearer $TOKEN" "$URL/api/skill" --create-dirs \
  -o ~/.hobkins/$ID/skills/hearthboard/SKILL.md
echo '{"idle_prompt": "Run the hearthboard wait loop (GET /api/wait) and do what it says.", "idle_seconds": 60}' \
  > ~/.hobkins/$ID/config.json
hobkin $ID

Each worker keeps the board's skill in its own skills/ folder, not _shared/: an agent that isn't a worker shouldn't see the board as its work system. The skill reads the token from HEARTHBOARD_TOKEN, and each worker has its own in secrets.env. The idle prompt keeps it checking for work, and it can also hold /api/wait open itself inside a command. Give each worker its own token, so claims and comments show which agent did what.

Releasing

Releases go to PyPI from GitHub Actions with Trusted Publishing (no token stored):

1. Bump __version__ in hobkin.py, commit and push.
2. Publish a GitHub release tagged vX.Y.Z (matching __version__). The workflow runs the tests on Python 3.9, 3.12 and 3.14, builds, checks the tag matches, and uploads to PyPI.

Running the workflow by hand (Actions → Publish → Run workflow) uploads to TestPyPI instead, for a dry run.

Measuring behaviour

The tests check that hobkin works; tests/eval_hobkin.py checks how well an agent behaves on it, so changes to AGENTS.md or the nudges can be measured, not eyeballed. It runs a real model (it costs calls; free models are fine) through six scenarios with a tiny work system (a queue, a team chat, a skill) and scores what it does:

Most scenarios also check that it didn't ask what to do and kept its workspace tidy.

python3 tests/eval_hobkin.py --model nvidia/nemotron-3.5-super-vl:free \
    --key-file ~/openrouter.key --runs 3            # --only queue,hello for some

It prints a line per run and a score per check across runs (e.g. queue 2/3 reported in the team chat). Run it before and after a change.

Tests

python3 tests/test_hobkin.py

These start real hobkin processes against tests/fake_api.py: a local stand-in for both APIs that rejects malformed conversations. So the tests cost nothing and don't need a key. They cover both API styles:

• commands, secrets, and hiding the model key from the shell;
• live and queued messages, the lock, /interrupt, messages arriving mid-command;
• stopping and resuming, and recovery after a crash mid-command;
• compaction, learning the window from an error, and the newest request surviving compaction;
• the shell's saved state (variables, arrays, functions, aliases, PATH, folder, login profile, reset), with secrets never saved;
• the file tools (exact text with quotes, $ and backslashes; repeated and missing matches; pasted line numbers; paging) and the skill tool;
• skills (listing, which copy wins, front matter);
• requests_per_minute shared across two agents;
• reminders (firing while idle, arriving mid-command with that command's result, validation, list and cancel, the prompt section, late and repeating after a restart, a broken reminders.json).