Skip to main content

Skipper

A desktop interaction tool that allows AI agents like Claude Code or OpenAI Codex to control web browsers and interact with desktop applications through natural language commands.

Overview

Skipper enables AI agents to:

  • View and navigate the web using your own browser
  • Navigate to URLs
  • Execute mouse clicks, keyboard input, and scrolling actions
  • [SOON] Interact with any desktop application through natural language prompts

The Vision

Skipper is a command line tool that gives AI agents the ability to interact with your browser/desktop. Unlike all-in-one computer use tools, Skipper takes the unix philosophy of "do one thing and do it well". Specifically, it is designed to be the "hands" of the AI agent, instead of the "brain".

Our architecture is designed to be privacy-preserving in the future. If the tool ends up being useful, we have designed the architecture to be possible to run locally only. The only sensitive information that would go to the cloud would be in text to the LLM agent, which could be censored or modified as necessary for privacy.

Installation

Prerequisites

  • Python 3.9 or higher
  • Chrome/Chromium browser with remote debugging enabled
  • Gemini API key (for AI-powered interactions)
  • Either
    • A computer capable of running OmniParser
    • An API key for Skipper to run this stage remotely

Install Skipper

# Install
pip install skipper-tool

# (Optional) Install with local dependencies
pip install 'skipper-tool[local]'

Setup Chrome Remote Debugging

Currently, skipper uses Chrome DevTools Protocol (CDP) to interact with the browser. Support is planned for other browsers, or just using the desktop app directly. However, for now we recommend using Chromium-based browsers like Chrome, Chromium, Edge, Brave, etc.

  1. Start a compatible browser with remote debugging enabled:
# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222

# Linux
google-chrome --remote-debugging-port=9222

# Windows
"C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222
  1. Keep the browser running in the background while using skipper_tool. You will need to provide a Gemini API key, as well as a Skipper API key if you intend to use the hosted version of OmniParser. If you instead want to run OmniParser locally, you can leave the Skipper API key blank.

Setup Initial Configuration

# Create a configuration file
skipper init --config

# This will prompt for your Gemini and Skipper API keys and create ~/.skipperrc

To use OmniParser locally, modify the ~/.skipperrc file to point to your local OmniParser model. Download the model file from here.

Usage

Skipper provides three main commands for AI agents:

1. View Window State

skipper view

Returns the current state of the active browser window, including:

  • Page title and URL
  • A simplified HTML representation of the page

2. Navigate to URL

skipper navigate --url "https://example.com"

Navigates the browser to the specified URL and returns the new page state.

3. Execute Commands

skipper command --command_type <type> --prompt "<description>"

Execute actions on the current page:

  • Click: skipper command --command_type click --prompt "Click the login button"
  • Type: skipper command --command_type type --prompt "Enter username: john.doe<Enter>"
  • Scroll: skipper command --command_type scroll --prompt "Scroll down"

Integration with Claude Code

Skipper is designed to work seamlessly with Claude Code and other AI agents. To integrate it, simply usage information to your agent's configuration, such as CLAUDE.md or AGENT.md. For an example AGENT.md file, see EXAMPLE.AGENT.md.

Advanced Features

Debug Mode

Enable debug logging to save screenshots and detailed logs:

# Set debug folder in ~/.skipperrc
[debug]
enabled = true
folder = "/path/to/debug/folder"

# Or use environment variable
export SKIPPER_DEBUG_FOLDER="/path/to/debug/folder"

Local AI Models

For enhanced privacy, you can use local AI models:

# Install local dependencies
pip install 'skipper-tool[local]'

# Configure local model paths in ~/.skipperrc
[models]
yolo_model_path = "/path/to/local/model.pt"

Custom Configuration

Edit ~/.skipperrc to customize:

[models]
screenshot_model = "gemini-2.5-flash"
ui_element_model = "gemini-2.5-pro"

[browser]
cdp_url = "http://localhost:9222"
context_index = 0
page_index = 0

[ui_interaction]
click_delay_seconds = 1.0
scroll_distance = 600
mouse_scale_factor = 0.5

Troubleshooting

Common Issues

  1. Chrome not responding: Ensure Chrome is running with --remote-debugging-port=9222
  2. API key errors: Set GEMINI_API_KEY environment variable or add to ~/.skipperrc
  3. Permission errors: Check that Skipper has access to the browser and debug port
# Enable verbose logging
export SKIPPER_DEBUG_FOLDER="/tmp/skipper-debug"
skipper view

# Check logs in the debug folder
ls /tmp/skipper-debug/

Security Considerations

  • Skipper requires access to your browser and can execute actions on your behalf
  • API keys are stored locally in ~/.skipperrc
  • Debug mode saves screenshots locally - ensure the debug folder is secure
  • Only use with trusted AI agents

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests if applicable
  5. Submit a pull request

License

AGPL

Support

Metadata

Release files for skipper-tool 0.2.13

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for skipper-tool 0.2.13
File Size Uploaded Provenance Actions
skipper_tool-0.2.13.tar.gz 36.8 kB Without provenance Details for skipper_tool-0.2.13.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for skipper-tool 0.2.13
File Interpreter ABI Platform Actions
skipper_tool-0.2.13-py3-none-any.whl Python 3 none any Details for skipper_tool-0.2.13-py3-none-any.whl

Total release size: 83.9 kB

Release files / skipper_tool-0.2.13.tar.gz

Download URL skipper_tool-0.2.13.tar.gz
Size 36.8 kB
Tags Source
SHA-256 checksum
How to use checksums
2941806275eac2eac96f80b2706dc3b6794edc8a6812b9b25e394e857ff9a07f
BLAKE2b-256 checksum
How to use checksums
19d88d83faee718479c6f257f0c8a6284460fc5d3c16faedb6b51edd91b1a39c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.5.16

Release files / skipper_tool-0.2.13-py3-none-any.whl

Download URL skipper_tool-0.2.13-py3-none-any.whl
Size 47.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2d46c4a0c6135f63ba856e14ef2b233da2753cf857efe53f196ee9158bc5a079
BLAKE2b-256 checksum
How to use checksums
ec1b3b87d907a1f107c9b69bf99c54d856c93835e3ad7261f0b9dec9f2fd012b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.5.16

Security for skipper-tool 0.2.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page