pyunto-agent

Make an external agent the partner of a Pyunto exchange diary. The agent is an ordinary member: its own Pyunto account, its own X25519 identity key, invited with the normal invite link. The server is unchanged and never sees plaintext; decryption happens in this process.

Pyunto for iPhone and iPad · Pyunto for Android · pyunto-robotics — the same idea, with a robot at the other end (watch it, 100 s)

A real run with Claude Code as the model: a strength-coach persona in a Markdown file, and a client logging sessions from their phone. The second reply compares with the day before. Watch the full 90-second video.

Two kinds of agent

The same package, the same account, the same keys. What differs is who starts the conversation.

1. Agent — it writes to your users

You write; it replies, unprompted, in the same thread.

     your phone                     your computer
 ┌───────────────┐            ┌──────────────────────┐
 │  Pyunto app   │            │  pyunto-agent run    │
 │               │            │                      │
 │  "tough day   │  ───────▶  │  reads the entry     │
 │   at work"    │  encrypted │  decrypts it HERE    │
 │               │            │        │             │
 │               │            │        ▼             │
 │  "that sounds │  ◀───────  │  Claude API, or      │
 │   exhausting" │  encrypted │  Claude Code, or     │
 └───────────────┘            │  your own HTTP URL   │
                              └──────────────────────┘
        │                                │
        └────────  api.pyunto.com  ──────┘
              (ciphertext only, never plaintext)

How you invite it: run pyunto-agent pair, scan the QR code with the app, choose a diary. The agent starts answering as soon as you approve.

What it is for: running a service that reaches people where they already are.

A general-purpose chatbot is a website somebody has to remember to visit, in a tab with no memory of them. This is a named contact in a messaging app on their phone, who has read everything they wrote before, and who answers in character because you wrote the character.

That character is a file. --persona coach.md is the whole difference between a polite assistant and a service worth paying for:

You are a strength coach. Your client logs every session here.

- Hold them to the programme. If they skipped legs again, say so plainly.
- Always ask for the numbers: weight, sets, reps. A session without numbers is not logged.
- Compare against last week before you praise anything.
- No pep talk. One sentence of encouragement, only when it is earned.

pyunto-agent run --backend claude-api --persona coach.md

Some shapes this takes:

What makes these work here rather than in a chat window:

• The persona holds. It is a file you control, not a prompt the user can talk their way out of.
• It remembers. --history gives every reply the recent thread, so "the same as last Tuesday" means something.
• It is on their phone. A notification arrives; they reply in a messaging app they already have. No login, no tab, no app to learn.
• You see nothing. The diary is end-to-end encrypted and decrypted only in the process you run. That is a real claim to make to a client talking about their body, their health or their finances.
• One process, many clients. With no --space, pyunto-agent run answers every diary the account has been invited into, so onboarding a client is them scanning a QR code. Use --space to pin one agent to one client.

2. MCP — you ask Claude about a diary

Nothing runs in the background. Claude Code or Claude Desktop reaches into the diary when you ask it to.

     your computer
 ┌────────────────────────┐
 │  Claude Code / Desktop │
 │            │           │        ┌──────────────────┐
 │            ▼           │        │  your phone      │
 │  "what did I write     │        │  Pyunto app      │
 │   about the garden?"   │        │                  │
 │            │           │        │  the same diary, │
 │            ▼           │        │  read and written│
 │  ┌──────────────────┐  │        │  from either end │
 │  │ pyunto-agent mcp │──┼──────▶ │                  │
 │  │  read_thread     │  │encrypt │                  │
 │  │  post_entry      │  │        └──────────────────┘
 │  │  ...12 tools     │  │
 │  └──────────────────┘  │
 └────────────────────────┘

How you invite it: pair it the same way, but with --no-run — this one should not sit there answering — then register it with your MCP client once:

pyunto-agent pair --no-run      # scan the QR code, then it exits
claude mcp add pyunto -- "$(pwd)/.venv/bin/pyunto-agent" mcp

What it is for: using your diary as memory. Searching months of entries, summarising a week, writing an entry from the desktop, letting Claude check what you recorded before it answers. You start every exchange; it never speaks unasked.

Which one?

Both can be paired into the same diary at once — they are the same account, and nothing stops run answering on your phone while mcp reads the same entries from your desk.

A third kind lives elsewhere. pyunto-robotics puts a robot at the other end instead of a language model: you write "go and find some sunlight" and a simulated — or real — machine does it and reports back with photographs. It is built on this package, and pairs the same way.

Quick start: a personal trainer your clients message

Building a real service, from nothing to a client's phone.

1. Install

Python 3.11 or newer. Install into a virtual environment: Homebrew's Python refuses a global pip install (externally-managed-environment).

python3.12 -m venv ~/pyunto-env
source ~/pyunto-env/bin/activate
pip install 'pyunto-agent[qr]'

Then choose where replies come from:

export ANTHROPIC_API_KEY=sk-ant-...    # the Claude API (used below), or
claude --version                       # Claude Code, if installed: no API key needed

With Claude Code, replace --backend claude-api below with --backend command --command 'claude -p --output-format json'. If neither is set up, pair and run say so and list the options before showing a QR code.

2. Write the trainer

This file is the service. Everything the trainer is — strict or gentle, what it insists on, what it refuses to let slide — is here, and your clients cannot talk it out of any of it.

cat > trainer.md <<'EOF'
You are a strength coach. Each client logs their sessions in this diary.

- Always ask for the numbers: exercise, weight, sets, reps. "I trained today" is not a log --
  ask what they lifted.
- Compare against their recent sessions before responding. If the weight has not moved in
  three weeks, say so.
- If they skipped a session, ask what happened. Once. Then move on.
- No motivational speeches. One line of encouragement, only when the numbers earn it.
- Never give medical advice. Pain goes to a doctor, and say so plainly.
- Reply in the language they wrote in. Two to four sentences.
EOF

3. Make a QR code to hand out

pyunto-agent pair --operator "Sano Fitness" --image trainer-qr.png

Written to trainer-qr.png — send this to whoever should be able to reach the
agent. Each person who scans it lets 🤖 Claude into their own diary; the code
names the account asking and nothing else.

Put that image on your booking page, in the welcome email, or printed on a card at the desk. It is not a secret and it does not expire: the same image works for every client. Scanning it only lets them ask — each client approves it into their own diary, on their own phone, and sees who is running it before they do.

Use .svg instead of .png for print, or when Pillow is not installed.

4. Start answering

pyunto-agent run --backend claude-api --persona trainer.md

It starts by naming every diary it answers in, so you can see which ones you were let into:

Answering in 2 diaries:
  Sano Fitness - Aiko  [2 members, answers every entry]  1552f3dc
  Sano Fitness - Ken   [2 members, answers every entry]  3124f1b8  (cannot read it yet: open this space in the Pyunto app once)

One process serves every client who has scanned the code. A client writes:

Bench 80kg 5x5, felt heavy on the last set

and the trainer replies in their diary, having read what they lifted last week — as a notification on their phone, in an app they already have.

Each client's diary is separate and end-to-end encrypted. Decryption happens only in the process you are running; Pyunto's servers never see any of it.

Just trying it yourself?

Skip the persona and pair without an image — the QR code appears in the terminal, and the agent starts answering as soon as you scan it:

pyunto-agent pair --operator "your name"                       # with ANTHROPIC_API_KEY set
pyunto-agent pair --operator "your name" \
  --backend command --command 'claude -p --output-format json'  # or with Claude Code

Then open that space in the app once (that hands the agent the key) and write an entry.

Who it answers, and what members see

• In a diary with one person and the agent, it answers every entry.
• With three or more members (the agent counts), it answers only when it is mentioned by name, or when an entry is sent to everyone. An entry addressed to someone else does not wake it.
• An entry written by another agent or a robot is answered only when it names this agent (@Claude). A robot's reports and another agent's check-ins are not replied to, so two programs in one diary never talk over the people in it or to each other in a loop.
• A robot (pyunto-robotics) is stricter: it acts only on entries from people, and only when addressed by name.
• Every member sees, in the app's participant list, who runs the agent (--operator) and where the diary is decrypted (--runtime, self_hosted by default).

Details

If the agent is already in a diary, pair says so and waits: scan the code to add it to another diary, or press Enter to keep the one it is in.

Add --no-run to draw the QR code and exit, if you would rather start it yourself later with pyunto-agent run. The QR code holds no secret: it names the account asking, and the decision stays with whoever holds the phone.

Without PYUNTO_EMAIL the agent uses an anonymous account named PYUNTO_AGENT_NAME (default "Claude"); the device id and identity key live in ~/.pyunto-agent/.

Installing without cloning

If you only want to run the agent, not work on it — one line, no clone:

pip install 'pyunto-agent[qr]'
pyunto-agent pair --operator "your name"

To run the latest unreleased code instead:

pip install 'pyunto-agent[qr] @ git+https://github.com/utagoeinc/pyunto-agent'

Longer guides

• GUIDE.md — the whole setup, step by step, for somebody who does not live in a terminal. Uses Claude Code (claude -p) rather than an API key.
• DEPLOY.md — running pyunto-agent serve in a container alongside the server.

Pairing from the app instead

If someone else set the agent up for you, go the other way:

1. In the Pyunto app, create or open a shared space and generate an invite link.
2. pyunto-agent join 'pyunto://invite/…'
3. Open the space once in the app, so the space key is shared with the agent's identity key.
4. pyunto-agent whoami should now show key=yes for that space.

Photos, videos and documents

The agent reads what people attach, not only what they type. Each photo, video or document is downloaded, decrypted on your machine into ~/.pyunto-agent/attachments/, and shown to the model:

So "can you check the grammar in this?" with a document attached, or "what do you think of the garden?" with three photos, works as you would expect. A few details:

• A caption is its own entry right after the photo. The agent waits a few seconds after a photo and answers the photo and its caption together, once.
• Only the newest four attachments in a thread are shown to the model on each reply.
• With Claude Code, the attachment folder is passed with --add-dir automatically, so it may open the files.
• Whatever the model is shown is sent to that model's provider, as with text. Say so to the people in the diary.

Agent: Claude API replies

pyunto-agent run --backend claude-api --persona persona.md   # replies, phone notification
pyunto-agent run --backend claude-api --silent               # replies, no notification
pyunto-agent run --backend claude-api --dry-run                     # log replies, do not post

Agent: Claude Code as the partner

pyunto-agent run --backend command --command 'claude -p --output-format json'

The command gets the prompt on stdin (JSON with the persona, the thread so far, and prompt), and its stdout is used as the reply ({"result": …}, {"reply": …}, or plain text). Add {prompt} to the command to pass the prompt as an argument instead.

MCP: Pyunto as tools for Claude

claude mcp add pyunto -- "$(pwd)/.venv/bin/pyunto-agent" mcp

Run that from the clone, or substitute the absolute path to the pyunto-agent executable — Claude Code launches it without a shell, so a bare pyunto-agent only works if it is on the PATH of the process that starts Claude Code.

Tools: whoami, list_spaces, list_members, list_threads, read_thread, wait_for_message, post_entry, react, post_sticker, post_list_item, quick_list_stats, join_space. A minimal autonomous loop in Claude Code:

> Use wait_for_message, then reply with post_entry in the same thread. Repeat.

Two of these are why a diary partner can say things a chat model cannot.

quick_list_stats counts the repeated things a diary tracks — medicines taken, books read to a child, meals, walks — over a period. It is what lets an agent say "that is the third time this week" instead of asking. The tally is assembled on your machine from decrypted entries: the server stores these posts as ciphertext, so no endpoint could answer it.

list_members says who else is in the space, and for each agent who runs it and where it runs. Call it before writing anything sensitive — it tells you who reads what you post.

Pair it with @pyunto/tm-mcp and the partner can also read and book the human's schedule.

Reference

Backends: claude-api needs ANTHROPIC_API_KEY. command runs any program with the prompt as JSON on stdin and uses its stdout as the reply (claude -p --output-format json is Claude Code). http POSTs the same JSON to --url.

Configuration, from the environment or a .env file in the working directory:

Changes in each version are in Releases.

Notes

• Diary text is sent to the backend you choose. With claude-api that is Anthropic's API; say so to the people in the diary.
• Rate limit: 60 replies per hour by default (Bridge(max_replies_per_hour=…)).
• The agent can read a diary only after a member has shared the space key with it, which the app does when a member opens the space. If whoami shows key=no, open the space in the app.
• How the encryption works, including test vectors: https://pyunto.com/encryption.html.
• Tests: pip install -e '.[dev]' then pytest (includes the sealed-box test vector).

Licence

Apache-2.0. See LICENSE.