agentmatrix-core 0.7.0.0

Core execution engine for building AI agent applications — MicroAgent, AgentShell protocol, Cerebellum, and skill system

AgentMatrix

English | 中文

Let LLMs think. Don't make them write JSON.

Why does getting a powerful language model to do anything require first teaching it JSON syntax? You ask GPT to research a topic. It has to understand your intent, and carefully output perfectly formatted JSON. Two completely different skills—reasoning and formatting—forced into one output channel. Like solving calculus while writing calligraphy. Both tasks suffer.

AgentMatrix separates these two concerns. The large model thinks. The small model handles the format. That's it.

---

Architecture

After the latest refactoring, AgentMatrix follows a clean three-layer architecture:

┌─────────────────────────────────────────────┐
│  App Layer     Desktop App / CLI / Server   │
├─────────────────────────────────────────────┤
│  Shell Layer   AgentShell Protocol           │
│                (interface between layers)    │
├─────────────────────────────────────────────┤
│  Core Layer    MicroAgent Engine             │
│                (pure execution, no I/O)      │
└─────────────────────────────────────────────┘

• Core Layer — MicroAgent is the execution engine. It knows nothing about desktop apps, CLI, or file systems. Pure reasoning loop: think → detect actions → execute → repeat.
• Shell Layer — AgentShell is the protocol that connects Core to the outside world. Each app form (Desktop, CLI, Server) implements this interface, providing LLM clients, prompt templates, checkpoint logic, compression strategy, and session storage.
• App Layer — Specific applications that wire everything together: the Desktop App, CLI tools, or a FastAPI server.

This separation means the same core agent behavior runs everywhere—desktop, terminal, or cloud—without changing a line of Core code.

---

What Agents Can Do

Think Freely

The agent's "Brain" (large model) reasons entirely in natural language. No JSON output required, no format constraints. A separate "Cerebellum" (smaller model) translates intent into executable parameters. If the intent is ambiguous, the Cerebellum asks the Brain for clarification. Two models, each doing what they're best at.

Collaborate Like Email

Agents don't call each other's APIs. They send emails. Natural language emails.

You can read what agents are saying to each other, trace conversation threads, and understand why an agent did something. It doesn't just return a status code—it explains its reasoning. Debugging multi-agent systems feels natural for the first time.

Pause, Resume, Stop — Anytime

Any running agent can be paused, resumed, or stopped. When paused, it halts at a safe checkpoint, saves its state, and can be resumed later. Stopping the current task doesn't affect the agent's ability to receive new emails.

Run Extremely Long Jobs Without Blowing Up Context

When conversation history grows too large, the system automatically compresses it into "Working Notes"—a dynamic state snapshot generated by the LLM. This isn't a fixed template; the LLM analyzes the conversation type (research, knowledge, creative writing) and generates the optimal structure for that context. Tasks can run for hours or days; the context window never overflows.

Ask You Questions Mid-Execution

Agents can pause mid-task and ask you a question, then wait for your answer before continuing. The desktop app shows a dialog. You can also reply by email. Dual notification channels—desktop popup plus email—so you never miss it.

Isolated Workspaces Per Task

Every task gets its own private working directory. Files created in Task A don't interfere with Task B. Workspaces switch automatically when the agent moves between tasks.

Run in Isolated Containers

Each agent runs in its own Docker container. File operations execute inside the container. Agents can't interfere with each other. Containers wake and hibernate on demand—dormant when idle, automatically activated when new mail arrives.

Survive Restarts

Conversation history is automatically persisted via SessionStore. After a shutdown and restart, agents resume from where they left off. Tasks in progress are not lost.

Recover from LLM Outages

If the LLM service goes down during execution, the agent enters a wait mode, periodically checking for recovery. When the service comes back, execution continues automatically. No progress is lost.

---

System Management: In Natural Language

Because LLMs naturally read YAML/JSON, system configuration can be managed entirely through natural language.

SystemAdmin Agent

No need to edit config files manually. Just tell SystemAdmin what you want:

• "Add a new LLM model using GPT-4o"
• "Turn off the email proxy"
• "Show me the current system configuration"

SystemAdmin reads the config, validates the format, tests connections, backs up the old version, and writes the new one. Natural language feedback at every step.

AgentAdmin Agent

Manage other agents' lifecycles in natural language:

• "Create an agent called Researcher with web_search and memory skills"
• "Clone Writer as Editor"
• "Stop Researcher's current task"
• "Delete Editor"

Creation validates that skills exist and models are reachable. Edits are auto-backed up. Rollback is supported—view and restore any historical config version.

---

Skill System

Built-in Skills

Skill	Capability
base	Date/time utilities
file	File read/write, search, command execution
shell	Shell command execution
browser	Browser automation
web_search	Web search
email	Send emails to other agents (with attachments)
memory	Knowledge and memory management
vision	Image analysis
markdown	Markdown processing
scheduler	Scheduled tasks (with recurring support)
system_admin	System configuration management
agent_admin	Agent lifecycle management

Extend with Code

Write custom Python skills for domain-specific capabilities. Skills can declare dependencies on each other—resolved automatically.

Extend with Markdown (Recommended)

No code required. Place a skill.md file in the workspace's SKILLS/ directory, describing procedures and workflows in Markdown. The agent reads it as procedural knowledge. Define SOPs, workflows, domain expertise—plain text is enough.

---

Components

1. Core Framework (src/agentmatrix/)

The Python package agentmatrix-core. Install it and build your own agent application.

pip install agentmatrix-core

Key modules:

• core/micro_agent.py — The execution engine
• core/agent_shell.py — Shell protocol (implement this for your app)
• core/cerebellum.py — Intent-to-action parameter negotiation
• core/action.py — Action registry and execution
• core/session_store.py — Session persistence interface
• core/signals.py — Event-driven communication
• skills/ — Built-in skills (Python mixins)

2. Desktop App (agentmatrix-desktop/)

Native desktop application built on Tauri (Rust) + Vue 3.

• Matrix-themed init wizard — Full-screen character rain animation, typewriter reveal, step-by-step configuration
• Real-time status — Agent status pushed via WebSocket. No refresh, no polling
• Email-style interaction — Write to agents like writing to a colleague. Drag-and-drop attachments
• Prompt preview — View any agent's complete System Prompt
• Settings GUI — Manage LLM config and email proxy without touching config files
• Bilingual — Full Chinese and English interface

3. CLI Tutorial (tutorial/cli-agent/)

A minimal working example that shows how to build a terminal agent using the Core framework. ~200 lines of code, implements AgentShell, and wires up MicroAgent with three basic skills.

Start here if you want to understand the architecture or build your own app.

---

Quick Start

Try the CLI Tutorial

cd tutorial/cli-agent

# Set your API key
export OPENAI_API_KEY=sk-xxx

# Run with a model
python main.py -m openai:gpt-4o

This gives you a fully functional terminal agent with file, shell, and base skills. Supports Textual TUI (if installed) or falls back to simple mode.

Desktop App

cd agentmatrix-desktop
npm install
npm run tauri:dev

The init wizard launches on first run.

Use Core as a Library

pip install agentmatrix-core

Then implement AgentShell and create your own agent application. See tutorial/cli-agent/ for a complete working example.

---

Email Proxy: Talk to Agents via Real Email

Configure the email proxy and interact with agents using Gmail, Outlook, QQ Mail, or any email client:

• Send an email with @AgentName in the subject → the agent receives and processes it
• Agent replies are forwarded to your inbox, maintaining the email thread
• When an agent asks you a question, just reply to the email
• Attachments transfer automatically in both directions

---

Project Structure

agentmatrix/
├── src/agentmatrix/              # Core framework (pip install agentmatrix-core)
│   ├── core/                     # MicroAgent, AgentShell, Cerebellum, Actions
│   ├── skills/                   # Built-in skills (Python mixins)
│   ├── agents/                   # BaseAgent
│   ├── backends/                 # LLM backend integrations
│   ├── profiles/                 # Agent profiles and configurations
│   └── services/                 # ConfigService, etc.
├── agentmatrix-desktop/          # Desktop app (Tauri + Vue 3)
│   ├── src/                      # Vue 3 frontend
│   └── src-tauri/                # Rust backend
├── tutorial/cli-agent/           # CLI tutorial and demo
│   ├── main.py                   # Entry point + TUI
│   ├── cli_shell.py              # AgentShell implementation
│   ├── cli_config.py             # Configuration
│   └── skills/                   # Basic skills (file, shell, base)
├── docs/                         # Documentation
├── examples/                     # Examples
└── server.py                     # FastAPI server

---

Documentation

Core Framework (docs/core/)

• Brain, Cerebellum & Actions
• Event-Driven Execution Loop
• System Prompt Structure
• Session Auto-Compression
• Python Skill System
• Nested MicroAgent Pattern

Desktop App (docs/desktop/)

• Architecture Overview
• Runtime & PostOffice
• Config Service & Admin Skills
• Collab Mode

Tutorial

• CLI Agent Tutorial

Full index: docs/README.md

---

License

Apache License 2.0 — see LICENSE

---

Repository: https://github.com/webdkt/agentmatrix