cua-operator

Computer-Using Agent (CUA) Operator

CUA is a new and evolving technology, but it currently lacks good documentation, mature libraries, and easy-to-use tools. While some automation or remote control libraries can work with CUA, they are often hard to set up. This project aims to make it simple for anyone to try out CUA without needing to understand it's internal details or complex configurations. It supports both Windows and macOS, and lets you connect your personal computer to a CUA compatible endpoint and explore the potential of CUA more easily.

Quick Start

Use CUA on your personal computer

Get Python Ready

pip install cua-operator openai

Run the following Python script and start your journey.

from cua_operator.bridges import OpenAICuaBridge
from cua_operator.operators import LocalCuaOperator
from openai import OpenAI

client = OpenAI(
    api_key="{OPENAI_API_KEY}"
)

operator = LocalCuaOperator(bridge=OpenAICuaBridge(client))
operator.run()

Customization

CUA-Operator is built around two core concepts: operators and bridges. An operator defines how you intend to interact with the CUA, whether through a local machine, a remote device, or potentially a Docker instance. A bridge specifies the service endpoint you connect to, such as OpenAI, with potential support for alternatives like Claude.

All available operators are in the cua_operator.operators namespace.

• Local CUA Operator: LocalCuaOperator
• Remote CUA Operator: RemoteCuaOperatorClient and RemoteCuaOperatorServer

All available bridges are in the cua_operator.bridges namespace.

• OpenAI CUA Bridge: OpenAICuaBridge

Samples: samples