Skip to main content

AbstractAssistant

AbstractAssistant is a gateway-native desktop assistant for the AbstractFramework ecosystem: a macOS menu-bar app with a compact palette, a hands-free voice conversation mode, and a small CLI.

The desktop client stays thin. AbstractGateway owns workflow discovery, durable runs, provider connections, tool execution and policy, workspace policy, and multimodal defaults. The assistant keeps only local state: your chats, window placement, downloads, and the overrides you choose for this app.

Tray / Palette / CLI -> AbstractGateway -> AbstractRuntime -> AbstractCore -> Providers

What you get

  • A menu-bar palette whose title tells you what the app is doing, with a session switcher fed by the gateway (your sessions from every device, optionally every client's) and a live gateway connection orb.
  • A local AbstractAvatar portrait as the voice control beside the composer (it ships with every install), with design and motion controls in Appearance. It follows listening, speech, tools, waits and errors; until you enable it, the control shows a simple sound icon.
  • Automations: schedule a conversation to run on the gateway at a fixed UTC interval or when an email arrives (with the result emailed to you if you want), read its runs as a chat, switch it off or on (Active), run it now, edit or archive it, answer the runs that wait for you, discuss a result, and get a tray notification only when a result is notable, a run fails or a run waits for you. See docs/automations.md.
  • What waits for you, everywhere: the app follows your event stream on the gateway, so the attention badge beside the session control and the menu-bar badge show the approvals, questions and new automation results across every conversation, within seconds. A click on the badge opens exactly what it counts. On one Mac you get one notification per item: the gateway's tray icon notifies first and the Assistant stays silent for what it already showed.
  • Live run activity in the transcript: the current step, elapsed time, recent tool calls with their arguments and durations, and pause / resume / stop controls.
  • Live replies: with Stream replies on (the default), the answer appears while the model writes it, with its reasoning folded away under Thinking; the finished answer replaces the live text.
  • Tool approvals in a modeless sheet that shows the gateway's risk tier for each call and never blocks the palette; "always allow on this Mac" and per-chat trust for low-risk tools.
  • Voice from the orb beside the composer: click it (or press ⌘⇧V) for a voice conversation — listen, send, speak, listen again — with Listening, Thinking or Speaking shown above it; right-click it to dictate into the message box. Spoken replies stream from the gateway.
  • A Settings window with seven sections: Connection, Models & reasoning (workflow, reasoning effort, reply streaming), Voice, Workspace (my default workspaces, this chat), Tools & permissions, Appearance & window, About. Every value says whether it is the gateway default or this app's override. About is also on the menu-bar icon (About AbstractAssistant…).
  • Local, persistent overrides for every route the assistant drives (chat, voice, image, video, music, sound) that ride each request without touching the gateway's shared defaults.
  • Per-answer statistics (tokens, tools, files, duration, model) with clickable detail views.
  • One reconstruction of every run: on AbstractGateway 0.17.0 or later the desktop app, assistant run and the web Assistant (web/) render a run from the gateway's replay view (GET /runs/{id}/view and its stream): status, replies, tool rows and the approval or question wherever it waits come from the gateway, so an approval answered on another client closes here too. On an older gateway the app folds the run's ledger itself and says so.
  • Multi-attachment composer with an image gallery, drag and drop, and artifact previews.
  • In your pocket and on your wrist: AbstractAssistant for iPhone and Apple Watch is one app (0.17.0, pre-release through TestFlight): the iPhone app carries the watch app, each signs in on its own and works without the other, and notifications reach you while the app is closed (Apple push, once your gateway has a push forwarder). The iPhone has the same conversations, tools and automations, with tab badges for what needs you (phone/iphone/README.md); the watch talks to the same gateway by voice around the orb, answers approvals and questions, continues your sessions and runs your automations, and your avatar on the watch face shows what needs you (watch/apple/README.md).

What's new

0.18.1 moves the voice control into the composer: the animated orb beside the message box starts and ends a voice conversation and shows whether the assistant is listening, thinking or speaking. Read aloud recovers by itself after a failed check and reads folder paths and the signs ≥, ≤ and ≠ as words. On Apple Watch, Home is the conversation screen and captures Notes, and Watch and iPhone can keep their sign-in when you disconnect (next TestFlight build). See the 0.18.1 release notes.

0.18.0 ships the activity avatar with every install, folds paused automations under "Paused · N" and archived ones under "Archived · N" in the switcher, and the activity avatar looks toward the pointer only when its design has eyes. The desktop palettes are now the kit's current ones (ui-kit 0.9.14). See the 0.18.0 release notes.

0.17.0 adds the optional desktop activity avatar (the avatar extra), automations that open as the chat with a sectioned editor, tool approvals in words, Off / Ask / Auto tool settings, and the web Assistant's Avatar conversation view. See the 0.17.0 release notes.

Install

pip install "abstractassistant[voice]"

The activity avatar ships with every install (since 0.18.0); turn it on in Settings → Appearance. The avatar extra is kept so older install lines still work; it adds nothing.

The voice extra adds local microphone capture (dictation and voice conversations). The base install covers text chat, spoken replies and gateway-backed media; STT and TTS run on the gateway either way.

Requirements: Python 3.10+, an AbstractGateway you can reach (0.16.0 or later for the voice library and the chat-wide tool policy; an older gateway keeps the labeled local grant and refuses the voice library's requests; 0.19.0 or later for read marks shared across your devices, without which conversation cards show no unread marks). AbstractCore is installed as a dependency (its minimum version is set in pyproject.toml). macOS is the primary tray target; Linux and Windows may work but are not packaged to the same standard.

Quick start

Start a local gateway for development:

export ABSTRACTGATEWAY_FLOWS_DIR="$PWD/abstractgateway/flows/bundles"
export ABSTRACTGATEWAY_AUTH_TOKEN="your-shared-token"
abstractgateway serve --host 127.0.0.1 --port 8080

Launch the tray assistant:

assistant

Run a single terminal turn:

assistant run --prompt "Search the web for the latest OpenAI news and summarize it with sources."

Connection overrides:

assistant --gateway-url http://127.0.0.1:8080 --gateway-token "$ABSTRACTGATEWAY_AUTH_TOKEN"

--gateway-url on its own is enough when you have already signed in to that gateway: the app reuses the sign-in saved in ~/.abstractassistant/gateway_connection.json.

If the gateway runs the AbstractGateway console, the simplest way to start the assistant is Open on its card there: the console launches the app already signed in as you, and the app remembers that sign-in for later launches. The console can sign in only an Assistant it launches, so quit a running Assistant before you click Open.

Each turn runs the gateway's default workflow for the assistant (abstractassistant.agent.v1). When the gateway sets none, the app runs its built-in orchestrator, the abstractassistant-orchestrator workflow it publishes to your tenant catalog. You can pick another assistant workflow in Settings → Workflow; the list holds only the assistant workflows the gateway says you can run (what your admin made available, plus your own: GET /api/gateway/bundles?executable_for=abstractassistant.agent.v1).

Defaults and durability

The gateway is the source of truth for the published workflow, provider and model defaults, tool inventory and approval defaults, workspace policy, run history, waits and artifacts.

The desktop client stores under ~/.abstractassistant/:

  • preferences.json — this app's overrides and preferences (model routes, reasoning effort, reply streaming, workspace grant, voice options, tool modes, hotkey, window size)
  • gateway_connection.json — gateway URL and sign-in state
  • session_cache.json, sessions/ — a rebuildable cache of the gateway's sessions (local labels, cached transcripts, each session's workspace); the sessions themselves live on the gateway
  • automations_notified.json — which automation notifications were already shown; automations themselves live on the gateway
  • downloads and cached audio

Documentation

Start with docs/README.md.

Development

pip install -e ".[dev]"
QT_QPA_PLATFORM=offscreen python -m pytest tests/basic -q

Metadata

Release files for abstractassistant 0.18.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for abstractassistant 0.18.1
File Size Uploaded Provenance Actions
abstractassistant-0.18.1.tar.gz 1.9 MB With provenance Details for abstractassistant-0.18.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for abstractassistant 0.18.1
File Interpreter ABI Platform Actions
abstractassistant-0.18.1-py3-none-any.whl Python 3 none any Details for abstractassistant-0.18.1-py3-none-any.whl

Total release size: 3.6 MB

Release files / abstractassistant-0.18.1.tar.gz

Download URL abstractassistant-0.18.1.tar.gz
Size 1.9 MB
Tags Source
SHA-256 checksum
How to use checksums
f97257e90c08e5013bb14da299af20b309c6cccae5eb9a1e045b8759fc827b3f
BLAKE2b-256 checksum
How to use checksums
c3f443b322dda3a99eeb36a4b61fd32e63b8650177c8a1f7a8e50985f1136065
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Release files / abstractassistant-0.18.1-py3-none-any.whl

Download URL abstractassistant-0.18.1-py3-none-any.whl
Size 1.7 MB
Tags Python 3
SHA-256 checksum
How to use checksums
9346c4477ee6d89d2a8b4edfdde89ef75db5d79572c8ab45fffc167c5c9dfd74
BLAKE2b-256 checksum
How to use checksums
bc90b9e581105905ee8175c8db7ce3d4d0467918715d1c4403927cae1cd809ad
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Security for abstractassistant 0.18.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for abstractassistant-0.18.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.18.1 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page