webtap

Browser debugging via Chrome DevTools Protocol with native event storage, browser-level multiplexing, and declarative target watching.

Features

• Browser-Level Multiplexing - Single WebSocket per Chrome port, multiple sessions
• Watch Model - Declarative watch/unwatch with auto-reattach on navigation, reload, SW restart
• Native CDP Storage - Events stored exactly as received in DuckDB per target
• Auto-Watch Children - Popups and new tabs opened by watched targets are automatically watched
• Smart Filtering - Built-in filters for ads, tracking, analytics noise
• MCP Ready - Tools and resources for Claude/LLMs
• Expression Evaluation - Pre-imported Python libraries for data extraction

Prerequisites

Chrome or Chromium with DevTools Protocol support:

# macOS
brew install --cask google-chrome

# Arch Linux
yay -S google-chrome

# Ubuntu/Debian
sudo apt install google-chrome-stable

Installation

# Install via uv tool (recommended)
uv tool install webtap-tool

# Or with pipx
pipx install webtap-tool

# Update to latest
uv tool upgrade webtap-tool

Quick Start

# 1. Install webtap
uv tool install webtap-tool

# 2. Optional: Setup helpers (first time only)
webtap setup-browser       # Install browser wrapper for debugging
webtap install-extension   # Install Chrome extension

# 3. Launch Chrome with debugging
webtap run-browser         # Or manually: google-chrome --remote-debugging-port=9222

# 4. Start webtap REPL (auto-starts daemon)
webtap

# 5. Watch and explore
>>> targets()                                          # List Chrome targets
>>> watch(["9222:abc123"])                             # Watch by target ID
>>> network()                                          # View network requests
>>> network(url="*api*")                               # Filter by URL pattern
>>> request(42, "9222:abc123", ["response.content"])   # Get response body

MCP Setup for Claude

# Quick setup with Claude CLI
claude mcp add webtap -- webtap

Or manually configure Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "webtap": {
      "command": "webtap"
    }
  }
}

MCP mode is auto-detected when stdin is piped (no flags needed).

Usage

Interactive REPL

webtap                     # Start REPL (or MCP server when piped)

CLI Commands

webtap --help              # Show help
webtap --version           # Show version
webtap status              # Daemon and connection status
webtap run-browser         # Launch Chrome/Edge with debugging
webtap setup-browser       # Install browser wrapper script
webtap install-extension   # Install Chrome extension
webtap setup-android -y    # Configure Android debugging
webtap daemon start|stop|status  # Manage daemon

Command Reference

Core Commands

Target Discovery & Watching

targets()                        # List all Chrome targets
watch(["9222:abc123"])           # Watch targets by ID (auto-attaches)
watching()                       # Show watched targets with state
unwatch(["9222:abc123"])         # Unwatch specific target
unwatch()                        # Unwatch all
navigate("https://...", "9222:abc123")  # Navigate to URL
reload("9222:abc123")            # Reload page
back("9222:abc123")              # Go back

Multi-Target Support

Watch multiple targets across Chrome instances simultaneously:

targets()                        # Shows all targets from all registered ports
watch(["9222:abc123"])           # Watch desktop Chrome page
watch(["9224:def456"])           # Watch Android Chrome page
watching()                       # Show all watched targets

Target IDs use format {port}:{short-id}. Child tabs opened by watched targets are automatically watched.

Network Monitoring

network()                              # Filtered network requests (default)
network(show_all=True)                 # Show everything (bypass filters)
network(status=404)                    # Filter by HTTP status
network(method="POST")                 # Filter by HTTP method
network(resource_type="xhr")           # Filter by resource type
network(url="*api*")                   # Filter by URL pattern
network(target="9222:abc123")          # Filter to specific target

Request Inspection

# Get HAR request details by row ID and target from network() output
request(42, "9222:abc123")                           # Minimal view
request(42, "9222:abc123", ["*"])                    # Full HAR entry
request(42, "9222:abc123", ["request.headers.*"])    # Request headers only
request(42, "9222:abc123", ["response.content"])     # Fetch response body (auto-decoded from base64)
request(42, "9222:abc123", ["request.postData", "response.content"])  # Both bodies

# With Python expression evaluation
request(42, "9222:abc123", ["response.content"], expr="json.loads(data['response']['content']['text'])")
request(42, "9222:abc123", ["response.content"], expr="BeautifulSoup(data['response']['content']['text'], 'html.parser').title")

Code Generation

# Generate Pydantic models from response bodies
to_model(42, "models/user.py", "User", "9222:abc123")
to_model(42, "models/user.py", "User", "9222:abc123", json_path="data[0]")

# Generate TypeScript/Go/Rust/etc types
quicktype(42, "types/user.ts", "User", "9222:abc123")
quicktype(42, "api.go", "ApiResponse", "9222:abc123")

Filter Management

filters()                                           # Show all filter groups
filters(add="myfilter", hide={"urls": ["*ads*"]})  # Create filter group
filters(enable="myfilter")                          # Enable group
filters(disable="myfilter")                         # Disable group
filters(remove="myfilter")                          # Delete group

Request Interception

fetch()                                    # Show capture state + rules
fetch({"capture": False})                  # Disable body capture
fetch({"mock": {"*api*": '{"ok":1}'}, "target": "9222:abc123"})  # Mock
fetch({"block": ["*tracking*"], "target": "9222:abc123"})        # Block

Console & JavaScript

console()                           # View console messages
console(level="error")              # Filter by level
entry(5, "9222:abc123", ["*"])      # Console entry details

js("document.title", "9222:abc123")                # Evaluate JavaScript
js("fetch('/api').then(r=>r.json())", "9222:abc123", await_promise=True)
js("var x = 1; x + 1", "9222:abc123", persist=True)  # Global scope
js("element.offsetWidth", "9222:abc123", selection=1)  # Selected element

clear()                             # Clear events
clear(events=True, console=True)    # Clear everything

Architecture

Daemon with Browser-Level Multiplexing

REPL / MCP Client (webtap)
    | JSON-RPC 2.0 (localhost:37650/rpc)
WebTap Daemon (background process)
    +-- FastAPI Server + RPCFramework
    +-- BrowserSession (one WebSocket per Chrome port)
    |       +-- CDPSession (per target, DuckDB storage)
    |       +-- CDPSession (per target, DuckDB storage)
    +-- ConnectionManager (per-target state)
    |
Service Layer (WebTapService)
    +-- NetworkService - Request queries via HAR views
    +-- ConsoleService - Message handling
    +-- FetchService - Request interception + body capture
    +-- DOMService - Element selection

How It Works

1. Browser-level WebSocket - One connection per Chrome port, multiplexed sessions
2. Watch model - Targets auto-reattach on navigation, reload, SW restart
3. Events stored as-is - No transformation, full CDP data preserved in DuckDB
4. HAR views pre-aggregated - Network requests correlated for fast querying
5. Base64 auto-decoded - Response bodies decoded to UTF-8; binary stays base64
6. On-demand body fetching - Response bodies fetched only when requested
7. Auto-watch children - Popups/new tabs from watched targets are automatically watched

Examples

Watch Targets and Monitor Traffic

>>> targets()
## Targets

| Target      | Type | Title           | URL                              | State   |
|:------------|:-----|:----------------|:---------------------------------|:--------|
| 9222:abc123 | page | GitHub          | https://github.com/angelsen      |         |
| 9222:def456 | page | YouTube Music   | https://music.youtube.com/       |         |

>>> watch(["9222:abc123"])
## Watching

| Target      | State    | Title  | URL                         |
|:------------|:---------|:-------|:----------------------------|
| 9222:abc123 | attached | GitHub | https://github.com/angelsen |

Inspect Network Requests

>>> network(url="*api*")
## Network Requests

| ID   | Target      | Method | Status | URL                            | Type  | Size |
|:-----|:------------|:-------|:-------|:-------------------------------|:------|:-----|
| 42   | 9222:abc123 | GET    | 200    | https://api.github.com/graphql | Fetch | 22KB |

>>> request(42, "9222:abc123", ["response.content"], expr="json.loads(data['response']['content']['text'])")
{'viewer': {'login': 'octocat', 'name': 'The Octocat'}}

Generate Types from API Responses

>>> to_model(42, "models/github.py", "GitHubResponse", "9222:abc123")
Model written to models/github.py

>>> quicktype(42, "types/github.ts", "GitHubResponse", "9222:abc123")
Types written to types/github.ts

Advanced Usage

Expression Evaluation

The request() command supports Python expressions with pre-imported libraries:

# Libraries: json, re, bs4/BeautifulSoup, lxml, jwt, yaml, httpx, urllib, datetime
request(42, "9222:abc123", ["response.content"], expr="json.loads(data['response']['content']['text'])")
request(42, "9222:abc123", ["response.content"], expr="BeautifulSoup(data['response']['content']['text'], 'html.parser').find_all('a')")

Browser Element Selection

Use the Chrome extension to select DOM elements, then access them:

selections()                                    # View all selected elements
selections(expr="data['selections']['1']")     # Get element #1 data
js("element.offsetWidth", "9222:abc123", selection=1)  # JS on selected element

Chrome Extension

Install the extension:

webtap install-extension   # Installs to platform data directory

Then load as unpacked extension in chrome://extensions/ with Developer mode enabled.

Design Principles

1. Store AS-IS - No transformation of CDP events
2. Query On-Demand - Extract only what's needed
3. Watch Model - Declarative target management with auto-reattach
4. Browser-Level Multiplexing - Single WebSocket per port
5. HAR-First - Pre-aggregated views for fast network queries

Requirements

• Chrome/Chromium with debugging enabled (--remote-debugging-port=9222)
• Python 3.12+
• Dependencies: websocket-client, duckdb, replkit2, fastapi, uvicorn, beautifulsoup4

Documentation

• Vision - Design philosophy
• Architecture - Implementation guide
• Tips - Command documentation and examples

Development

git clone https://github.com/angelsen/tap-tools
cd tap-tools
uv sync --package webtap
uv run --package webtap webtap

make check         # Type check
make format        # Format code
make lint          # Fix linting

Daemon uses ports 37650-37659 (auto-discovery). See ARCHITECTURE.md for details.

License

MIT - see LICENSE for details.