cua-agent is a general Computer-Use framework for running multi-app agentic workflows targeting macOS and Linux sandbox created with Cua, supporting local (Ollama) and cloud model providers (OpenAI, Anthropic, Groq, DeepSeek, Qwen).
Get started with Agent
Install
pip install "cua-agent[all]"
# or install specific loop providers
pip install "cua-agent[openai]" # OpenAI Cua Loop
pip install "cua-agent[anthropic]" # Anthropic Cua Loop
pip install "cua-agent[omni]" # Cua Loop based on OmniParser (includes Ollama for local models)
pip install "cua-agent[ui]" # Gradio UI for the agent
Run
async with Computer() as macos_computer:
# Create agent with loop and provider
agent = ComputerAgent(
computer=macos_computer,
loop=AgentLoop.OPENAI,
model=LLM(provider=LLMProvider.OPENAI)
# or
# loop=AgentLoop.ANTHROPIC,
# model=LLM(provider=LLMProvider.ANTHROPIC)
# or
# loop=AgentLoop.OMNI,
# model=LLM(provider=LLMProvider.OLLAMA, model="gemma3")
)
tasks = [
"Look for a repository named trycua/cua on GitHub.",
"Check the open issues, open the most recent one and read it.",
"Clone the repository in users/lume/projects if it doesn't exist yet.",
"Open the repository with an app named Cursor (on the dock, black background and white cube icon).",
"From Cursor, open Composer if not already open.",
"Focus on the Composer text area, then write and submit a task to help resolve the GitHub issue.",
]
for i, task in enumerate(tasks):
print(f"\nExecuting task {i}/{len(tasks)}: {task}")
async for result in agent.run(task):
print(result)
print(f"\n✅ Task {i+1}/{len(tasks)} completed: {task}")
Refer to these notebooks for step-by-step guides on how to use the Computer-Use Agent (CUA):
- Agent Notebook - Complete examples and workflows
Using the Gradio UI
The agent includes a Gradio-based user interface for easy interaction. To use it:
# Install with Gradio support
pip install "cua-agent[ui]"
# Create a simple launcher script
```python
from agent.ui.gradio.app import create_gradio_ui
app = create_gradio_ui()
app.launch(share=False)
The Gradio UI provides:
- Selection of different agent loops (OpenAI, Anthropic, OMNI)
- Model selection for each provider
- Configuration of agent parameters
- Chat interface for interacting with the agent
You can also embed the Gradio UI in your own application:
# Import directly in your application
from agent.ui.gradio.app import create_gradio_ui
# Create the UI with advanced features
demo = create_gradio_ui()
demo.launch()
# Or for a simpler interface
from agent.ui.gradio import registry
demo = registry(name='cua:gpt-4o')
demo.launch()
Agent Loops
The cua-agent package provides three agent loops variations, based on different CUA models providers and techniques:
| Agent Loop | Supported Models | Description | Set-Of-Marks |
|---|---|---|---|
AgentLoop.OPENAI |
• computer_use_preview |
Use OpenAI Operator CUA model | Not Required |
AgentLoop.ANTHROPIC |
• claude-3-5-sonnet-20240620• claude-3-7-sonnet-20250219 |
Use Anthropic Computer-Use | Not Required |
AgentLoop.OMNI |
• claude-3-5-sonnet-20240620• claude-3-7-sonnet-20250219• gpt-4.5-preview• gpt-4o• gpt-4• phi4• phi4-mini• gemma3• ...• Any Ollama-compatible model |
Use OmniParser for element pixel-detection (SoM) and any VLMs for UI Grounding and Reasoning | OmniParser |
AgentResponse
The AgentResponse class represents the structured output returned after each agent turn. It contains the agent's response, reasoning, tool usage, and other metadata. The response format aligns with the new OpenAI Agent SDK specification for better consistency across different agent loops.
async for result in agent.run(task):
print("Response ID: ", result.get("id"))
# Print detailed usage information
usage = result.get("usage")
if usage:
print("\nUsage Details:")
print(f" Input Tokens: {usage.get('input_tokens')}")
if "input_tokens_details" in usage:
print(f" Input Tokens Details: {usage.get('input_tokens_details')}")
print(f" Output Tokens: {usage.get('output_tokens')}")
if "output_tokens_details" in usage:
print(f" Output Tokens Details: {usage.get('output_tokens_details')}")
print(f" Total Tokens: {usage.get('total_tokens')}")
print("Response Text: ", result.get("text"))
# Print tools information
tools = result.get("tools")
if tools:
print("\nTools:")
print(tools)
# Print reasoning and tool call outputs
outputs = result.get("output", [])
for output in outputs:
output_type = output.get("type")
if output_type == "reasoning":
print("\nReasoning Output:")
print(output)
elif output_type == "computer_call":
print("\nTool Call Output:")
print(output)
1 maintainer
Metadata
1 maintainer
Release files for cua-agent 0.1.23
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cua_agent-0.1.23.tar.gz | 91.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cua_agent-0.1.23-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 214.3 kB
Release files / cua_agent-0.1.23.tar.gz
| Download URL | cua_agent-0.1.23.tar.gz |
|---|---|
| Size | 91.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
da6eac0a333c97c57d25cdc0d9de50fe01fe6cc68a16655012536099c1e5d4f4
|
|
BLAKE2b-256 checksum How to use checksums |
5cf1aa7ad9911e43e9e19c6b32cdb54527ed7ac89a750c729e48bf7dda5a2831
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.11
|
Release files / cua_agent-0.1.23-py3-none-any.whl
| Download URL | cua_agent-0.1.23-py3-none-any.whl |
|---|---|
| Size | 122.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
aac6e50246f84e5f5f315ec864eca2df36659fd0045f03de123348c3d5e7f30d
|
|
BLAKE2b-256 checksum How to use checksums |
3ce1e856cc496c1cdc74a23dca16a0a2d2d25b3756c72c1f973667e6fd41011a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.11
|
