agent-webui 0.41.0

Agent Web Interface for Pydantic AI Agents

Agent WebUI

Version: 0.41.0

Documentation — The architecture, graph-activity event model, feature surface, and ecosystem integration of Agent WebUI are maintained in the official documentation, which presents the project's reference material as a searchable, published site.

A React-based chat interface for Pydantic AI Agents built using Agent Utilities.

Built with Vercel AI SDK and designed to work with Pydantic AI's streaming chat API.

Overview

Agent WebUI is a highly interactive, responsive chat interface designed specifically for complex agentic workflows. It visualizes background reasoning, renders tool invocations, and provides a real-time sideband view of parallel multi-agent graph execution.

Table of Contents

• Overview
• Features
• Architecture
• Installation
• Usage
• Development
• Detailed Documentation
• Documentation
• License

Features

Core Chat Experience

• Streaming message responses with reasoning display
• Tool call visualization with collapsible input/output
• Interactive Elicitation Forms for structured user input
• Conversation persistence via localStorage and server-side storage
• Dynamic model and tool selection
• Dark/light theme support
• Mobile-responsive sidebar

Advanced Agent Capabilities

• Scheduling jobs with cron task monitoring
• Memory management with timeline visualization and importance scoring
• MCP Support with unified specialist discovery
• Multi-model support with dynamic routing
• Multi-modal support -- image attachments can be sent alongside text messages for visual reasoning
• Mermaid diagram rendering via Streamdown

Graph Orchestration & Visualization

• Graph Activity Visualization -- real-time specialist tracking with domain routing, parallel execution status, tool calls, and expert reasoning displayed in a collapsible timeline (GraphActivity.tsx)
• Enhanced GraphView -- interactive graph visualization with:
  • Force-directed, hierarchical, and circular layout algorithms
  • Zoom/pan controls and PNG export
  • Node type filtering and detailed node inspection
  • Real-time graph statistics and relationship explorer
• Human-in-the-loop tool approval -- security-sensitive tool calls are intercepted and require explicit user permission via an inline approval card (ApprovalCard.tsx)

Knowledge Management

• Knowledge Base Management -- comprehensive KB system with:
  • Document ingestion from multiple sources (PDF, DOCX, EPUB, Markdown, URLs)
  • Article CRUD with concept extraction and fact indexing
  • Health check monitoring with contradiction detection
  • Hybrid search across knowledge bases
• Memory Management -- agent memory system with:
  • Timeline visualization of memory creation and updates
  • Importance scoring and temporal decay tracking
  • Tag-based organization and advanced search
  • Impact analysis for code changes
• Brain & Knowledge Orthogonal Views -- policy-guided retrieval across:
  • Semantic view (vector-based similarity)
  • Temporal view (episodic memory with decay)
  • Causal view (reasoning traces and "why" links)
  • Entity view (people, organizations, code symbols)

Ontology Operator Views

Three operator views surface the agent-utilities ontology system (kg.ontology) directly in the UI. All data is fetched live (no fixtures) through the /api/enhanced/ontology/* backend routes, which resolve against the same IntelligenceGraphEngine backend the rest of the app uses and enforce the fine-grained permissioning gate (kg.ontology.permissioning.enforce).

• Object Explorer (ObjectExplorerView.tsx) -- faceted/full-text search and property filters over object sets, with a results table, pivot across a link type, search-around (related objects N hops away), aggregate metrics (count/sum/avg/min/max, optionally grouped), bulk Actions on a row selection (run through the governed ActionExecutor with Edit-Ledger writeback and HITL approval for high-risk verbs), and save/list of object sets.
• Object View renderer (ObjectView.tsx) -- renders a single ontology object as the central hub: value-type-aware properties, computed derived properties (visually distinguished), in/out links grouped by link type and navigable, marking/permission badges, a bitemporal edit-history timeline, and inline edit/revert affordances. Two form factors via variant (full page, panel embeddable) and two layout sources via layout (standard, derived from the object type's interface schema; configured, a stored widget composition).
• Vertex scenario view (VertexView.tsx) -- a graph-canvas exploration over an object-set seed with link-type expansion (search-around), per-node derived-property computation, and a what-if scenario mode: remove nodes from the working set, recompute an aggregate scenario metric against the reduced id set, and compare it to the unmodified baseline (delta).

/api/enhanced/ontology/* backend routes

Defined in agent/agent_webui/api_extensions.py:

Method & path	Purpose
GET /ontology/object-types	Distinct object/node types (registry types + interface implementers)
GET /ontology/property-types	Property-type registry (KG-2.47)
GET /ontology/interfaces	Interfaces with their implementers (KG-2.38)
GET /ontology/interfaces/{name}/implementers	Implementers of a single interface
POST /ontology/object-set/search	Resolve an object set by ids / kind / property filters / query
POST /ontology/object-set/search-around	Related objects N hops from a seed set
POST /ontology/object-set/pivot	Pivot an object set across a link type, grouped
POST /ontology/object-set/aggregate	Aggregate (count/sum/avg/min/max), optionally grouped
POST /ontology/object-set/save	Persist a named/saved object set
GET /ontology/object-set/list	List saved object sets
GET /ontology/actions	Registered OntologyActions, optionally scoped to a type
POST /ontology/object-set/action	Apply a bulk action via the governed executor (writeback + HITL approve)
GET /ontology/object/{object_id}	Full object view: properties, links, derived, markings, history, layout
POST /ontology/object/{object_id}/edit	Record a durable edit (property_set / link_add / link_remove)
POST /ontology/object/{object_id}/revert	Revert an edit via a compensating edit
POST /ontology/function/invoke	Invoke a typed, versioned ontology function (audited runtime, KG-2.41)
POST /ontology/derive	Compute a single derived property for an object (KG-2.40)
POST /ontology/document/process	Process a document into Document + Chunk objects (KG-2.48)
GET /ontology/object-view/{object_type}	Get a type's ObjectView: stored (configured) else standard (schema)
POST /ontology/object-view/{object_type}	Save a configured ObjectView widget composition for a type

Unified Agent Homelab

• Ecosystem Service Discovery -- Automatically scans for and lists active/installed local agent packages and MCP servers.
• 5-Domain Navigation Layout:
  • DevOps & Workspace: Workspace Matrix code status, branch operations, and git pulling.
  • Brain & Knowledge: Visual SVG graph nodes, Cypher console, scientific literature papers explorer, and Prompts visual configurator.
  • Infrastructure Hub: SSH tunnels drawer, remote terminal commands, CPU/RAM/Disk dials, active process trees, and Docker registry boards.
  • Lifestyle & Automation: SmartHome device dimmers, Calendar tasks, qBittorrent torrent speedometers, and MealieScaled grocery trackers.
  • System Config: Scheduled crons timeline and Global settings configurator forms.

Spec-Driven Development (SDD)

• Constitution Management -- project governance and tech stack configuration
• Specification Management -- user stories, acceptance criteria, and requirements
• Implementation Planning -- technical approach with dependency mapping
• Task Management -- parallel execution tracking with status updates
• Memory Synchronization -- automatic capture of SDD lifecycle to knowledge graph

Resource Management

• MCP Tool Discovery -- automatic discovery and registration of MCP server tools
• A2A Agent Registry -- peer-to-peer agent communication and coordination
• Specialist Spawning -- dynamic creation of specialized sub-agents with curated toolsets
• Resource Explorer -- unified view of all callable resources (MCP tools, A2A agents, skills)

Workspace Management

• Files -- browse and manage workspace files with upload/download
• Skills -- view and configure universal skills
• Scheduling -- monitor and manage cron tasks
• Configuration -- adjust agent and workspace settings
• Knowledge -- manage knowledge base and embeddings
• Graph -- interactive knowledge graph visualization and exploration
• Memory -- agent memory management with timeline and search
• SDD -- spec-driven development lifecycle management

Agent Identity & Context

• Agent identity display in sidebar with workspace-aware context
• Multi-agent team support with P2P messaging
• Workspace-specific configuration and preferences

Architecture

Protocol Support

• AG-UI (default): Standard Vercel AI SDK streaming via /api/chat. Supports text, reasoning, tool calls, and graph sideband events. Uses the @ai-sdk/react useChat hook for real-time streaming.
• ACP (opt-in): Advanced Agent Communication Protocol via /acp/*. Provides session management, planning modes, and approval bridges. Enabled by setting VITE_ENABLE_ACP=true. Routes through the full HSM graph pipeline via create_graph_acp_app(), ensuring ACP clients benefit from specialist routing, parallel execution, circuit breakers, and verification.

Backend Integration & Centralized Gateway

The WebUI backend (agent/agent_webui/server.py) integrates with a Centralized Epistemic Gateway hosted directly in agent-utilities (agent-utilities-kg). This gateway centralizes database connections, memory stores, and workspace actions for multiple concurrent agents and client applications.

1. Vercel AI SDK Integration: Mounts Pydantic AI's streaming chat routes for /api/chat.
2. Centralized Routing & API Proxy: Proxy-routes /api/enhanced/* dynamically to the Centralized Epistemic Gateway:
   • Knowledge Graph APIs: Memory CRUD, node linking, search, impact analysis, Cypher console queries.
   • Knowledge Base & Ingestion: Resource parsing, sitemap scanning, literature exploration, hybrid vector search.
   • SDD Lifecycle APIs: Constitution, specification matrices, implementation planning, and automatic task/memory sync.
3. Unified Tools Configuration (Centralized /tools & /tools/toggle):
   • Lists and configures all 3 tool classes (MCP Servers, Built-in Tools, Agent Skills/Workflows/Graphs) via standard /tools API.
   • Toggles status of individual skills, workflows, graphs, servers, or native tools via /tools/toggle, persisting preferences directly inside the Knowledge Graph as Preference nodes.
4. Symmetric Bilateral Graph Execution Routes:
   • Exposes 7 high-fidelity graph REST routes (/graph/query, /graph/search, /graph/write, /graph/ingest, /graph/analyze, /graph/orchestrate, /graph/configure) that map symmetrically to the Knowledge Graph's registered tools, letting external applications execute graph methods as standard HTTP endpoints.
5. Unified Specialist Discovery: Consolidates MCP specialists and A2A peers into a single discovered registry at graph boot.
6. Robust Storage Abstraction: Utilizes the gateway's optimized engine database connection pooling and backend adapters (LadybugDB, Neo4j).

All communication is fully traceable, logging session parameters, agent identities, and provenance for complete ecosystem visibility.

Unified Discovery Architecture

graph LR
    subgraph Sources ["Discovery Sources"]
        NA["NODE_AGENTS.md<br/>(MCP Specialists)"]
        A2A["A2A_AGENTS.md<br/>(Remote Peers)"]
    end
    subgraph Unified ["Unified Discovery (a2a.py)"]
        DAL["discover_all_specialists()"]
        DS["list[DiscoveredSpecialist]"]
    end
    subgraph Graph ["Graph Bootstrap"]
        TagPrompts["tag_prompts"]
        StepDescs["get_step_descriptions()"]
        Nodes["Specialist Nodes"]
    end
    NA --> DAL
    A2A --> DAL
    DAL -->|Deduplicated by tag| DS
    DS --> TagPrompts
    DS --> StepDescs
    DS --> Nodes

Both MCP and A2A specialists are registered through the same code path. The frontend does not need to distinguish between them -- it consumes identical sideband events (specialist_enter, tools-bound, subagent_completed) regardless of specialist source.

Key Frontend Components

Component	File	Responsibility
Chat.tsx	src/Chat.tsx	Main chat interface with streaming, tool execution, graph activity, multi-modal input, approval workflows
GraphActivity.tsx	src/components/GraphActivity.tsx	Real-time graph execution timeline showing routing decisions, parallel execution, tool binding, and expert reasoning
ApprovalCard.tsx	src/components/ApprovalCard.tsx	Human-in-the-loop tool approval card for security-sensitive operations
Part.tsx	src/Part.tsx	Message part renderer handling text, tool calls, elicitation forms, sources, and images
app-sidebar.tsx	src/components/app-sidebar.tsx	Navigation sidebar with conversation history, agent identity, and view switching
Graph Views
GraphView.tsx	src/components/views/GraphView.tsx	Interactive graph visualization with layouts, zoom/pan, node inspection, and statistics
Knowledge Management
KnowledgeBaseView.tsx	src/components/views/KnowledgeBaseView.tsx	Knowledge base ingestion, article management, health checks, and search
MemoryView.tsx	src/components/views/MemoryView.tsx	Memory CRUD with timeline visualization, importance scoring, and advanced search
Ontology Operator Views
ObjectExplorerView.tsx	src/components/views/ObjectExplorerView.tsx	Object-set search, filters, pivot, search-around, aggregate, bulk Actions, and save/list
ObjectView.tsx	src/components/views/ObjectView.tsx	Single-object hub: properties, derived props, links, markings, edit-history timeline, inline edit/revert
VertexView.tsx	src/components/views/VertexView.tsx	Graph-canvas object-set exploration with link expansion and what-if scenario metrics (baseline vs delta)
SDD Lifecycle
SDDView.tsx	src/components/views/SDDView.tsx	Spec-driven development: constitution, specs, plans, tasks, and memory synchronization
Workspace Views
FilesView.tsx	src/components/views/FilesView.tsx	Workspace file browser with upload and download
SkillsView.tsx	src/components/views/SkillsView.tsx	Universal skills viewer and configuration
SchedulingView.tsx	src/components/views/SchedulingView.tsx	Cron task monitoring and management
ConfigurationView.tsx	src/components/views/ConfigurationView.tsx	Agent and workspace configuration
KnowledgeView.tsx	src/components/views/KnowledgeView.tsx	Knowledge base and embedding management (legacy)
Protocol Clients
acp-client.ts	src/lib/acp-client.ts	ACP protocol client for session management, RPC calls, and SSE event streaming
mcp-context.tsx	src/lib/mcp-context.tsx	MCP tool context provider for the React tree

State Management

• Server state: React Query (@tanstack/react-query) for workspace data, conversations, and configuration
• Chat state: Vercel AI SDK useChat hook for message streaming and tool execution
• Client state: React Context and local component state
• Persistence: localStorage for conversation IDs and preferences, server-side via /api/enhanced/chats

Installation

Ensure you have Node.js (with pnpm) and Python 3.11+ installed.

# Clone the repository
git clone https://github.com/pydantic/ai-chat-ui.git
cd ai-chat-ui

# Install frontend dependencies
pnpm install

# Install backend dependencies (in a virtual environment)
cd agent
uv venv
source .venv/bin/activate
uv pip install -e .
cd ..

Usage

Start both the frontend and backend servers to run the interface:

# Terminal 1: Start the Python backend
pnpm run dev:server

# Terminal 2: Start the Vite frontend dev server
pnpm run dev

Navigate to http://localhost:5173 to interact with the agent.

Development

pnpm install
pnpm run dev:server  # start the Python backend (requires agent/ setup)
pnpm run dev         # start the Vite dev server

Testing

# Frontend tests
pnpm run test              # Run unit tests
pnpm run test:coverage     # Run with coverage
pnpm run test:watch        # Watch mode
pnpm run test:e2e          # Run E2E tests with Playwright

# Backend tests
pytest agent/agent_webui/__tests__/              # Run backend tests
pytest agent/agent_webui/__tests__/ --cov        # With coverage

Environment Variables

Variable	Default	Description
VITE_ENABLE_ACP	false	Enable ACP protocol support alongside AG-UI

Documentation

The complete reference for Agent WebUI is published as a searchable site at the official documentation. It covers the system architecture and protocol flow, the agent and graph-activity event model, the feature surface and API endpoints, and the unified homelab ecosystem integration.

License

MIT