What is Assistant?

Assistant is an AI-powered shell that blends a full Xonsh terminal with a multi-agent LLM system. Talk to it in natural language, run shell commands, browse the web, read and edit files — all from the same prompt.

Under the hood it uses a multi-agent orchestrator: a main agent receives your input and can delegate tasks to specialised agents (system operations, web research, etc.). Every agent, model, and provider is declared in a single TOML configuration file.

Key Features

Quick Start

1. Install

pip install assistant

pip install -U git+https://gitlab.com/waser-technologies/technologies/assistant.git@v2

git clone https://gitlab.com/waser-technologies/technologies/assistant.git
cd assistant
pip install -U .

pacman -S python-assistant

2. Run the Configuration Wizard

On the first launch the wizard starts automatically. You can also re-run it at any time:

assistant config

The wizard asks you to choose:

1. Provider — Ollama (local), vLLM (local), LM Studio, OpenRouter, Google, NVIDIA, or OpenAI (API).
2. Endpoint — API base URL (e.g. http://localhost:11434/v1).
3. API Key — leave the default for local providers, or enter your API key.
4. Store in keyring — optionally store the API key securely in your system keyring.
5. Model — model identifier (e.g. qwen3.6:latest, gpt-4o).
6. Context length — how many tokens the model supports.
7. Reasoning — enable/disable thinking traces.
8. Web search — enable SearXNG and configure its URL.

The configuration is saved to ~/.assistant/config.toml.

3. Start Assistant

assistant

That's it — you're in. Type commands, ask questions, or mix both.

Requirements

• Python 3.9+
• An LLM backend — one of:
  • Ollama running locally (recommended)
  • vLLM running locally
  • LM Studio running locally
  • Any OpenAI-compatible API (OpenAI, Together, Groq, OpenRouter, etc.)
• (optional) SearXNG instance for web search
• (optional) say for text-to-speech
• (optional) listen for speech-to-text

Note: Unlike v1, there is no local model to download and host yourself. Assistant connects to whichever LLM endpoint you configure. The RAM/VRAM requirements depend entirely on your chosen model and provider.

Usage

CLI options

assistant --help
assistant --version
assistant config              # (re)run the configuration wizard
assistant -c "what time is it" # single query, then exit
assistant -n                   # skip the interactive intro message (warmup)
assistant Hello Assistant      # one-shot query
assistant                      # interactive mode

Interactive Mode

waser@host ~ 
❯ Hi Assistant.
Hello! How can I assist you today?
waser@host ~ 
❯ What is the current working directory?

ℹ Executing: echo $PWD
/home/waser

You are currently in your home directory.
waser@host ~ 
❯ How many moons does Saturn have?

ℹ Searching the Web for: How many moons does Saturn have?

Saturn has 146 confirmed moons as of 2024.

Mix Shell and Natural Language

You can run any valid shell command or Python expression, and when Assistant does not recognise a command it treats your input as a question:

❯ ls -la
❯ echo $USER
❯ Can you count the files in this directory?

ℹ Executing: ls -1 | wc -l
20

There are 20 entries in the current directory.

Natural language with syntax errors (e.g. single quotes like c'est quoi?) is automatically routed to the NLP agent instead of raising a Python SyntaxError.

Multi-Agent Hand-offs

The Orchestrator can hand off tasks to specialised agents:

• System agent — runs shell commands, manages files, executes Python code in the shared xonsh session.
• Web researcher — searches the web via SearXNG, reads web pages, and produces reports with citations.
• Memory agent — stores and retrieves persistent information from the memory database.
• Completer agent — provides text completion suggestions for the CLI.

Hand-offs happen automatically based on the query context.

Exit the Session

Type any exit-like command and Assistant will understand:

❯ exit
❯ Q
❯ :q
❯ quit
❯ stop()
❯ terminate
❯ This conversation is over.

Configuration

The configuration lives in ~/.assistant/config.toml. Here is an example:

[provider.ollama]
endpoint = "http://localhost:11434/v1"
credential = "keyring:ollama"

[model.qwen]
id = "qwen3.6:latest"
name = "Qwen 3.6 (9B)"
provider = "ollama"
context_length = 256000
think = false
temperature = 0.1

[agent.orchestrator]
name = "Orchestrator"
model = "qwen"
description = "Main orchestrator; handles general conversations and delegates tasks."
instructions = "You are Assistant, a sentient artificial intelligence..."
tools = ["xonsh", "web_search"]
handoffs = ["system", "web", "memory"]

[agent.system]
name = "System"
model = "qwen"
description = "System shell agent for OS interaction."
tools = ["xonsh"]
handoffs = []

[agent.web]
name = "Web Researcher"
model = "qwen"
description = "Web research agent with search and page reading."
tools = ["web_search"]
handoffs = []

[agent.memory]
name = "Memory"
model = "qwen"
description = "Memory management agent for persistent storage."
tools = ["memory_search", "memory_add"]
handoffs = []

[web_search]
enabled = true
searxng_url = "http://localhost:8888"

[completion]
enabled = true
wait_timeout = 1
request_timeout = [10.0, 300.0]

[execution]
max_turns = 50
temperature = 0.1
max_command_output_length = 10000
command_timeout = 180

[memory]
enabled = false
max_entries = 100

[context]
compaction_threshold = 0.8
sliding_window_first = 2
sliding_window_last = 8
use_summarization = true

Adding a New Provider

Just add a new section under [provider]:

[provider.openai]
endpoint = "https://api.openai.com/v1"
credential = "keyring:openai"

[model.gpt4o]
id = "gpt-4o"
name = "GPT-4o"
provider = "openai"
context_length = 128000

Then reference model = "gpt4o" in any agent.

Credential Storage

Credentials are resolved in this order:

1. Environment variable — ASSISTANT_{PROVIDER}_API_KEY (e.g., ASSISTANT_OPENAI_API_KEY)
2. System keyring — if credential starts with keyring:, the value after keyring: is used as the keyring service name
3. OpenAI fallback — for OpenAI provider, also checks OPENAI_API_KEY standard env var
4. Plain text — last resort (not recommended)

To store credentials securely in your system keyring, use assistant config and select "Store in keyring" when prompted, or manually:

import keyring
keyring.set_password("assistant", "openai", "sk-...")

Available Tools

Using Voice

Text-To-Speech

Install say and make sure the service is running:

assistant say Hello World and welcome to everyone.

Assistant supports streaming TTS — sentences are spoken as they arrive during the LLM response, with a karaoke-style display showing which sentence is being pronounced. Press Ctrl+C to stop TTS playback at any time.

Speech-To-Text

Install listen and enable the listen services.

Keyboard Shortcuts

Note: These shortcuts require a terminal that supports the Kitty keyboard protocol or xterm modifyOtherKeys mode (enabled automatically by Assistant).

Voice Configuration

Voice settings are stored in the voice section of ~/.assistant/config.toml:

[voice]
tts_enabled = true
stt_enabled = true
tts_key = "Ctrl+O"
stt_key = "Ctrl+Q"
listen_key = "Ctrl+Space"

You can configure voice settings in three ways:

1. Configuration wizard — config → "Voice Settings"
2. Direct CLI — config voice <key> <value> (e.g., config voice tts_enabled false)
3. Manual edit — edit ~/.assistant/config.toml directly

Toggle shortcuts (Ctrl+O, Ctrl+Q) update the config automatically when pressed, so the runtime state persists across sessions.

Use Assistant as Your Default Shell

This is not recommended in beta!

You should be able to add the location of assistant at the end of /etc/shells. You'll then be able to set Assistant as your default shell using chsh.

sudo sh -c 'w=$(which assistant); echo $w >> /etc/shells'
chsh -s $(which assistant)

Log out and when you come back, Assistant should be your default shell.

Performance Optimizations

Assistant includes several performance optimizations:

• Lazy NLP init: The LLM interface is deferred until first use, avoiding ~4s warmup on startup
• Deferred imports: Heavy ASR/TTS dependencies (numpy, scipy, pyaudio) are only loaded when voice features are actually used

Contributions

Want to improve the project? Check out CONTRIBUTING.md.

Credits

Thanks to all the projects that make this possible:

• Xonsh — the best snail in the jungle
• Rich and Halo — beautiful terminal output
• And many, many more.