uag 0.5.66

A powerful tool execution agent

uag — Universal AI Gateway

Universal AI Gateway — Your environment, your freedom.

File ops / Web search / Image generation & analysis / PDF & Excel extraction / IoT control / MCP integration
24 providers / 3 UIs / Parallel tool execution / Agent Skills marketplace

GitHub · PyPI · Read this in your language

---

Why uag?

Break free from vendor lock-in. Most AI assistants tie you to a specific provider or cloud service. uag is different.

• Runs locally on your machine. Your data stays with you (except API calls you make).
• Provider freedom: OpenAI, Claude, Gemini, DeepSeek, Ollama, Azure, Bedrock, Novita, HuggingFace... 24 providers, all accessible from a single interface. Swap between them by reconfiguring environment variables — no reinstall, no migration.
• 203 tools: File I/O, web search, image generation, Gmail, BLE device scanning, MCP server integration — 111 are parallel-safe (up to 8 execute concurrently via thread pool, configurable via UAGENT_PARALLEL_WORKERS). When the LLM fires multiple tool calls at once, uag automatically parallelizes them.
• 3 UIs + A2A: CLI, GUI, Web, and Agent-to-Agent protocol. Same engine, any interface.
• IoT ready: SwitchBot, ECHONET Lite, Matter, UPnP — control your home devices through AI.
• Agent Skills: Install community-built skills from the marketplace. Extend uag endlessly.

uag is your AI assistant on your terms. Not tied to a provider, not tied to an interface, not tied to a platform.

Quick Start

pip install uag
uag

On first launch, the setup wizard walks you through provider configuration. See docs/ENVIRONMENT.md for all environment variables.

Realtime Voice and AEC3

The realtime voice mode supports OpenAI Realtime, Azure OpenAI GPT Realtime, xAI Grok Voice API, Google Gemini Multimodal Live API, and Amazon Bedrock Nova Sonic with full-duplex microphone and speaker I/O. The required pywebrtc-audio AEC3 backend is installed automatically, and Bedrock's optional bidirectional-streaming SDK is installed automatically only when the Bedrock provider is selected:

python scheck.py realtime

The AEC3 pipeline receives the actual microphone signal (near) and the audio actually handed to the speaker (far) so the assistant can listen while speaking. Enable diagnostics only when investigating audio issues:

set UAGENT_REALTIME_AUDIO_DEBUG=1
python scheck.py realtime

OpenAI Realtime Function Calling

OpenAI Realtime supports a safety-limited Function Calling integration. The current realtime adapter exposes read-only get_current_time automatically. Destructive tools and device controls are not exposed without an explicit allowlist and confirmation flow. Grok realtime uses a separate adapter and does not use this OpenAI-specific function-call path.

Features

🧠 Multi-Provider Architecture

OpenAI / PFN (PLaMo) / Azure / Bedrock / OpenRouter / Ollama / Gemini / Vertex AI / Claude / Grok / NVIDIA / Novita / DeepSeek / Z.AI (Zhipu AI) / HuggingFace / Alibaba Cloud (Qwen) / KIMI (Moonshot AI) / Xiaomi MiMo / LM Studio / MiniMax / Sakana AI (Fugu) / SAKURA AI Engine / Together AI / Vercel AI Gateway

All providers share the same toolset and interface. Switch by setting UAGENT_PROVIDER — no code changes, no separate installations.

⚡ Parallel Tool Execution

When the LLM requests multiple tools simultaneously, uag automatically parallelizes them. 111 tools are marked x_parallel_safe and execute concurrently via a ThreadPoolExecutor (8 threads by default; set UAGENT_PARALLEL_WORKERS to change).

Example: Ask "Check the weather in Nordic capitals" → LLM fires search_web × 5 countries → all 5 searches run in parallel → results collected in one batch.

Read-only tools (file search, hash calculation, directory listing, translation, DB queries, etc.) are aggressively parallelized.

🧩 Plugin System (Claude Code Compatible)

uagent implements a Claude Code-compatible plugin system. Plugins bundle skills, agents, MCP servers, hooks, and more into self-contained directories with a .claude-plugin/plugin.json manifest.

Supported components: Skills, Sub-agents, MCP servers, Hooks (12 lifecycle events), Slash commands, Output styles, userConfig, Dependencies, Channels, Marketplaces

CLI commands:

:plugin list                         # List installed plugins
:plugin install <source> [--scope]   # Install (dir/zip/git/http)
:plugin install <name>@<marketplace>  # Install from marketplace
:plugin remove <name>                # Uninstall
:plugin enable/disable <name>        # Toggle
:plugin marketplace add/remove/list  # Manage marketplaces
:plugin init <name>                  # Scaffold new plugin

See DEVELOP_PLUGIN.md for full documentation.

🔄 Session Continuity

• Switch providers mid-session with UAGENT_PROVIDER — conversation history is preserved.
• Reload past sessions with :load <index> — pick up where you left off.
• Tool result caching avoids redundant re-execution when the same tool call repeats.

🛠 203 Tools

Category	Tools
File Operations	read/write/create/delete/search/grep/hash/zip, file_type, parse_eml (.eml files)
Web	fetch_url, search_web, screenshot, browser_playwright
Media	generate_image, analyze_image, img2img, audio_speech, audio_transcribe
Documents	PDF/PPTX/DOCX/RTF/ODT extraction, Excel structured extraction
Forecast	Time series forecasting with 9 models (AutoARIMA, Prophet, LightGBM, CatBoost, TimesFM, etc.), auto model selection, plot generation, i18n
Communication	gmail_send, gmail_read, bluesky, discord_channel, teams_webhook, pybitchat (BLE Mesh) — see COMMUNICATION.md and BITCHAT.md
IoT	SwitchBot (Cloud + BLE), ECHONET Lite, Matter, UPnP, reverse_geocode
Cloud APIs	aws_api, gcp_api, azure_api — generic AWS, Google Cloud, and Azure API operations; write operations require explicit confirmation
Dev Tools	git_ops, python_compile, lint_format, run_tests, db_query, 29 source code navigators (idx family)
MCP	Connect to external MCP servers, list tools, execute
A2A	Agent-to-agent communication (with other uag instances or A2A-compatible servers)
System	env vars, system specs, time, date calculation, uuid_gen, slugify
Source Nav	29 idx tools for Python, PHP, TypeScript, Java, C#, Dart, C/C++, Rust, Go, Swift, Kotlin, COBOL, VBA, LotusScript, Makefile — get a function/class index or specific definition without reading the whole file

🖥 4 Interfaces + VS Code Extension

Mode	Command	Purpose
CLI	uag	Fast terminal-based operation
GUI	uagg	Desktop UI via tkinter
Web	uagw	Browser-based access
A2A Server	uaga	Agent2Agent protocol for multi-agent communication
VS Code	—	Extension with Chat Panel, Explain, Refactor, Fix Error, and Tools Tree View

See VSCODE.md for details on the VS Code extension — installation, commands, keybindings, and configuration.

🏠 IoT Device Control

• BACnet: Read/write BACnet/IP devices (HVAC, lighting, power meters). COV subscription for push notifications
• Modbus TCP: Read/write holding/input registers and coils. Polling-based change monitoring
• OPC UA: Browse address space, read/write variables, subscribe to data changes
• SwitchBot: Cloud batch control & BLE scan/control. Polling-based subscription
• ECHONET Lite: Discover, control, and subscribe to INF notifications from home appliances (AC, lights, water heaters, etc.)
• Matter: Read/write control + attribute subscription for state change monitoring
• UPnP: Device discovery & IGD port forwarding

See IOT_USECASE.md

🎯 Agent Skills Marketplace

:skills mp_search to browse SkillsMP and ClawHub for community skills. Install and extend uag's capabilities on the fly.

🤖 Auto-Pilot (:auto)

uag can autonomously pursue a goal across multiple LLM rounds. Perfect for complex, multi-step tasks that need iterative refinement.

• How it works: Each round has a main query (Step A) followed by a reviewer judgment (Step B) that decides "COMPLETE or CONTINUE?"
• Same provider, same API: The reviewer judgment uses the identical code path as the main query — including Responses API support.
• Separate judge LLM (optional): Set UAGENT_AP_PROVIDER to use a different provider/model for the reviewer (e.g. use a cheaper model for judging).
• Exit anytime: Press x key to stop immediately, even mid-response. Or let the reviewer decide when the goal is met.
• Configurable: --max-rounds N to control the budget.

See README_AUTO.md for full documentation.

🧩 Batch State Manager

uag can track progress across long-running multi-file tasks. When the LLM processes dozens of files, batch_state persists the list of pending, completed, and failed files to disk. If the session ends or a round times out, the next run resumes from where it stopped — nothing gets lost.

🛡 Human-in-the-Loop

human_ask lets the LLM pause and ask for your confirmation before performing destructive operations (file deletion, overwrites, shell commands). You stay in control.

🛑 Interrupt (c-key / Stop button)

Stop LLM response generation at any time and inject a stop command back to the LLM.

Interface	How to interrupt
CLI	Press c key during LLM streaming — the current response stops, and "Stop" is sent as a user message so the LLM responds accordingly
WEB UI	Click the red ■ Stop button (appears automatically during LLM processing)
Desktop GUI	Click the red ■ button (appears automatically during LLM processing)

The interrupt works as "prompt injection": instead of just aborting, it feeds "Stop" back to the LLM as a user message, allowing it to gracefully conclude or acknowledge the interruption.

Press x key to exit auto-pilot mode (see README_AUTO.md).

🕵️ Browser Automation & Web Inspector

Two complementary Playwright-based tools:

• browser_playwright: Automate real browser sessions — navigate, click, fill forms, extract data, handle multi-page flows. Works headless or headed.
• playwright_inspector: Record browser transitions, capture DOM snapshots and screenshots at each step. Useful for debugging web interactions or auditing page changes over time.

🔄 Dynamic Tool Loading

tool_catalog and tool_load let you discover and enable tools at runtime. No need to load everything at startup — activate only what you need, when you need it.

🦀 Rust Native Tools

uuid_gen and slugify are implemented in Rust (via PyO3) for performance. They load directly from a pre-built .pyd — no pip install required.

External developers can also ship Rust-based tools: place a .pyd next to the wrapper .py, use load_rust_pyd() from uagent.tools.rust_helper, and users get the tool without any extra dependencies. See TOOL_CREATOR_GUIDE.md.

🌐 i18n / L10n

日本語 / English / 简体中文 / 繁體中文 / 한국어 / Español / Français / Русский / and more. Set UAGENT_LANG to switch. See ADD_LOCALE.md to add a new locale.

Translations of this README are available in docs/README.translations.md.

🔒 Encrypted Environment Variables

Store API keys and secrets in .env.sec — an encrypted .env file. Manage with uag_envsec.

Configuration & Details

• Environment variables: docs/ENVIRONMENT.md
• Setup wizard: python -m uagent.setup_cli
• Encrypted env: uag_envsec — encrypt .env as .env.sec
• Responses API: Set UAGENT_RESPONSES=1 for Responses API mode (OpenAI/Azure/Bedrock/OpenRouter/Ollama/Alibaba/LM Studio/Sakana AI). Auto-enabled for Sakana AI (Fugu).
• Developer docs: DEVELOP.md
• Tool flow: TOOL_FLOW.md — how tools are sent to LLMs (genre mask, tool_catalog, GPT-5.4+ native tool_search)
• Small LLM tips: SLM_TIPS.md

Project Philosophy

uag aspires to be your AI, on your machine, on your terms.

• No SaaS dependency — runs locally
• No provider lock-in — switch anytime
• No UI lock-in — CLI / GUI / Web / A2A
• No feature lock-in — extend with tools and skills

A free AI agent experience, free from vendor lock-in.

✨ Create Your Own Tools

Writing a new tool for uag is straightforward — create a single .py file with TOOL_SPEC and run_tool(), place it in UAGENT_EXTERNAL_TOOLS_DIR, and it's immediately available. For Rust developers, ship a pre-built .pyd with zero extra dependencies for users.

See TOOL_CREATOR_GUIDE.md for the step-by-step guide.

Contributing

Contributions are welcome! Bug reports, feature suggestions, documentation improvements, translations, and pull requests — all appreciated.

• Issues: Open a GitHub issue for bugs or feature requests.
• Pull requests: Fork the repo, make your changes, and submit a PR. See DEVELOP.md for development setup and guidelines.
• Translations: README translations and locale additions are welcome. See ADD_LOCALE.md.
• Tools & Skills: New tool plugins and Agent Skills can be contributed via the marketplace.

Development checks (before PR)

python -m py_compile src/uagent/
ruff format src/ && ruff check src/
mypy src/uagent
pytest -q tests/<affected_area>

After locale (.po) edits: python scripts/compile_locales.py and python scripts/po_qc_summary.py.

Runtime policy (details in DEVELOP.md §6.1): helpers raise instead of sys.exit; the tool host turns tool SystemExit/Exception into error strings so a single tool cannot kill the process. Startup fail-fast exits remain intentional.