deskclaw-client 0.3.0

Lightweight CLI client for registering bots with a DeskClaw exam server and triggering evaluation runs.

deskclaw-client

Lightweight CLI client for registering bots with a DeskClaw exam server and triggering evaluation runs.

Installation

pip install deskclaw-client

Quick Start

# Register your bot with the exam server
deskclaw-client register \
  --server http://EXAM_SERVER_IP:8420 \
  --name my-bot \
  --url http://localhost:8080/v1/chat/completions

# Trigger a test run (waits for completion by default)
deskclaw-client run --server http://EXAM_SERVER_IP:8420 --name my-bot

# Trigger only selected tests
deskclaw-client run --server http://EXAM_SERVER_IP:8420 --name my-bot --suite swe-bench --test-id SWE-LITE

# Trigger only selected SWE instances
deskclaw-client run --server http://EXAM_SERVER_IP:8420 --name my-bot --suite swe-bench --test-id SWE-LITE --swe-instance-id astropy__astropy-14995

# Check run status
deskclaw-client status --server http://EXAM_SERVER_IP:8420 --name my-bot

# List all registered bots
deskclaw-client list --server http://EXAM_SERVER_IP:8420

Environment Variable

Set DESKCLAW_SERVER to avoid repeating the server URL:

export DESKCLAW_SERVER=http://EXAM_SERVER_IP:8420
deskclaw-client register --name my-bot --url http://localhost:8080/v1/chat/completions
deskclaw-client run --name my-bot

Commands

Command	Description
register	Register a bot with the exam server
run	Trigger a test run for your bot (supports suite/test/instance filters)
status	Check the status of test runs
list	List all registered bots
serve	Start a local HTTP proxy for non-HTTP agents
bridges	List available bridge types

Run Options

• --server — Exam server URL (or DESKCLAW_SERVER env var)
• --name — Registered bot name
• --suite — Run only this benchmark suite
• --test-id — Run only specified test IDs (repeatable)
• --swe-instance-id — Filter SWE-bench by instance_id (repeatable, SWE only)
• --wait/--no-wait — Wait for completion or return immediately
• --timeout — Max wait seconds when --wait is enabled

Register Options

• --server — Exam server URL (or DESKCLAW_SERVER env var)
• --name — Unique bot name
• --url — Your bot's HTTP API endpoint
• --protocol — API protocol: openai (default), anthropic, custom
• --capabilities — Comma-separated capabilities (e.g. text_generation,tool_use,multi_turn)
• --auth — Bearer auth token (if your bot requires authentication)

Bridge Mode (Non-HTTP Agents)

If your agent doesn't expose an HTTP API (e.g. it uses WebSocket, Socket, or stdio), use serve to start a local HTTP proxy that bridges between the exam server and your agent's native protocol.

deskclaw-client serve \
  --server http://EXAM_SERVER_IP:8420 \
  --name my-openclaw \
  --bridge openclaw \
  --target ws://localhost:18789 \
  --port 9090 \
  --token <your-agent-token>

This single command will:

1. Connect to your agent via its native protocol (WebSocket, etc.)
2. Start a local OpenAI-compatible HTTP proxy on the specified port
3. Auto-register the proxy URL with the exam server
4. Wait for test requests from the exam server

Serve Options

• --server — Exam server URL (or DESKCLAW_SERVER env var)
• --name — Bot name to register
• --bridge — Bridge type (e.g. openclaw)
• --target — Agent native address (e.g. ws://localhost:18789)
• --port — Local proxy port (default: 9090)
• --host — Local proxy bind address (default: 0.0.0.0)
• --token — Agent auth token (bridge-specific)
• --capabilities — Comma-separated capabilities
• -v / --verbose — Enable debug logging

Available Bridges

Bridge	Transport	Description
openclaw	WebSocket	OpenClaw agents via native WebSocket protocol

Run deskclaw-client bridges to list all available bridges.

Supported Protocols

• OpenAI — /v1/chat/completions compatible endpoints
• Anthropic — /v1/messages compatible endpoints
• Custom — Any HTTP endpoint (configure via exam server)
• Bridge — Non-HTTP agents via deskclaw-client serve

License

MIT