Skip to main content

Octoparse Relay

Octoparse Relay connects an existing Playwright script to the Chromium-based browser profile where the Octoparse Relay extension is installed. Start relay-cli once to pair the browser and reuse its loopback Relay across Python runs. Without a usable CLI Relay, the SDK creates a temporary Relay as before.

Shared Relay for agents

Install the Python package into the tool or project environment to get the relay-cli entrypoint. With a matching extension built and loaded, run:

relay-cli start
relay-cli status

start launches a detached server, requests an available port, opens the browser activation page and waits for the extension handshake. It prints JSON containing the port and authenticated CDP URL. Calling it again reuses the running server.

Browser selection follows --browser > BROWSER > platform default:

relay-cli start --browser chrome

--browser accepts a controller name or %s command template supported by Python's webbrowser on the current platform. Explicit selection fails instead of falling back to a different browser. BROWSER retains standard-library candidate-list behavior. To change an already paired browser configuration, stop the Relay before starting it with the new choice.

The CLI atomically records its endpoint in the current user's cache:

  • macOS: ~/Library/Caches/octoparse-relay/daemon.json
  • Windows: %LOCALAPPDATA%/octoparse-relay/daemon.json
  • Linux: $XDG_CACHE_HOME/octoparse-relay/daemon.json (or ~/.cache/...)

Both connect() and connect_async() automatically discover and check this endpoint, then connect through the CLI's port without starting another server or opening another activation page. Existing scripts need no changes:

browser = playwright_relay.connect(p.chromium)

Set OCTOPARSE_RELAY_URL=<cdp_url> to override file-based discovery with a local or remote Relay CLI server, for example:

export OCTOPARSE_RELAY_URL='wss://relay.example.com/relay?role=client&token=YOUR_TOKEN'

Relay URLs accept ws:// or wss://, require /relay, role=client, and a nonempty token. The SDK checks /status on the same host and port using HTTP for WS or HTTPS for WSS. A reverse proxy must expose both /relay (WebSocket) and /status (HTTP) at the domain root. The remote browser must already be paired with that Relay. The CLI's loopback listener is unchanged; exposing the remote service through a proxy or tunnel is managed separately.

The SDK still checks Relay status and falls back to a temporary local Relay if unavailable or unpaired; a busy Relay raises an error. CDP connections use Playwright defaults unless overridden through **kwargs, which both connect() and connect_async() forward to connect_over_cdp(). no_defaults defaults to True and can be overridden. timeout is in seconds, controls temporary Relay pairing, and is multiplied by 1000 before being passed to CDP in milliseconds. For example, timeout=60 means 60 seconds for each stage. Pairing defaults to 15 seconds when timeout is omitted or None; CDP keeps its Playwright default. timeout=0 disables both limits. Each stage has its own wait limit, not a shared total deadline. Existing scripts keep their timeout values in seconds; explicit timeouts now also apply to the CDP connection. Loopback status requests bypass proxies; remote requests use the normal proxy settings. The former endpoint argument and OCTOPARSE_RELAY_ENDPOINT variable have been replaced by OCTOPARSE_RELAY_URL.

To connect directly to a standard Chromium CDP server instead:

export OCTOPARSE_CDP_URL='https://browser.example.com:9222'

Both connect() and connect_async() accept endpoint_url through **kwargs (connect(chromium, **kwargs) / connect_async(chromium, **kwargs)):

browser = playwright_relay.connect(p.chromium, endpoint_url="http://localhost:9222", timeout=30)
# Async: await playwright_relay.connect_async(p.chromium, endpoint_url="http://localhost:9222")

A nonempty endpoint_url takes priority over OCTOPARSE_CDP_URL. When omitted, None, empty, or whitespace-only, address selection uses OCTOPARSE_CDP_URL, then Relay discovery, then a temporary Relay. The selected URL is passed as the endpoint_url argument to connect_over_cdp(); other kwargs are forwarded. Both explicit endpoint_url and OCTOPARSE_CDP_URL bypass Relay discovery. HTTP(S) and WS(S) CDP endpoints are supported. This mode does not check local Relay ports, start a server, or open an extension activation page. Connection errors propagate without falling back to a local Relay. Closing the connection does not invoke any remote service stop/delete API.

Both URL environment variables are trimmed; empty or whitespace-only values are treated as unset. Priority is OCTOPARSE_CDP_URL > OCTOPARSE_RELAY_URL > the local discovery record > a temporary Relay. OCTOPARSE_RELAY_STATE_FILE overrides the discovery file location. Endpoint URLs contain credentials; treat them like passwords.

Close your own pages and Python connection after each run. Closing the client keeps the background server and extension connection alive. One Python client can use it at a time; a busy Relay reports an error instead of being replaced. Separate Python executions do not share variables or event listeners. An open Relay-owned page can be discovered again by the next client.

relay-cli stop

stop closes the server and extension pairing, preserving Chrome and its tabs. If the server or extension connection is unavailable, the SDK falls back to its temporary connection flow. Run start again to pair the background server. The persistent client capability requires rebuilding and reloading the extension.

For source-checkout use before reinstalling the package entrypoint:

uv run python -m octoparse.playwright_relay.cli start

start --no-browser starts only the listener, for transport diagnostics. The normal start command completes browser pairing. The CLI executes no Python code; agents explore and validate scripts through the native SDK.

Automatic AWS WAF CAPTCHA handling

Relay can resolve AWS WAF main-document image CAPTCHAs on Relay-owned tabs. The extension handles the page SDK, challenge state, answer validation and cookies; the Relay server calls an OpenAI-compatible vision model. There is no popup or model credential storage in the extension. Other CAPTCHA providers, third-party iframe CAPTCHAs and standalone Challenge solving are not supported. Detection observes responses after tab ownership is established; it does not reconstruct a CAPTCHA response that already occurred before an existing tab was adopted.

Configure a persistent server with:

relay-cli start --base-url https://provider.example/v1 --api-key "" --model vision-model

Both persistent and temporary servers read these environment variables:

export OCTOPARSE_RELAY_LLM_BASE_URL=https://provider.example/v1
export OCTOPARSE_RELAY_LLM_API_KEY=""
export OCTOPARSE_RELAY_LLM_MODEL=vision-model

For each field, a nonempty environment variable overrides the corresponding CLI argument; an empty variable falls back to the argument. The API base URL includes /v1 or the provider's equivalent prefix; Relay appends /chat/completions. The API key is optional. An empty key omits the Authorization header. The model must support multimodal image input and SSE chat-completion responses.

Configuration is captured when the server starts. Restart the persistent server to change it; a Python client borrowing an existing server uses that server's configuration, not the client's environment. Temporary servers read the environment of the Python process that creates them.

Missing or invalid model configuration does not prevent either server from starting. The model channel reports llm.error with Model request failed; the extension maps automatic CAPTCHA failures to Failed to solve CAPTCHA automatically. CDP and subsequent requests continue working. A returned model answer is not proof that WAF accepted it; the extension validates and submits it. Scripts should wait for the expected business page; automatic solving does not extend Playwright timeouts.

Only image data and model prompts are sent to the model service. Site cookies, tokens, vouchers and WAF verification state stay in the extension. Closing or navigating a tab, losing Relay ownership, disconnecting the client or stopping the server cancels its pending solving work. An already-started page SDK getToken() call cannot be forcibly cancelled and may still update the site's cookie; cancelled Relay work cannot submit a late model answer or install its final token.

The extension requires HTTP(S) host access and the webRequest, webNavigation, scripting, cookies and storage permissions in addition to its existing debugger and tab access. Runtime ownership checks restrict automatic solving to Relay-owned tabs; these checks do not narrow the install permissions. Reload the rebuilt extension after updating.

Agent skill

The Python package includes skills/octoparse-relay/SKILL.md starting with version 0.1.3. After installation, locate the bundled instructions with:

relay-cli skill

Ask your agent to run relay-cli skill and read the returned SKILL.md before using Relay. In a project managed by uv, use uv run relay-cli skill. The command also works in a source checkout. It does not start the server or open a browser. Agents that only discover skills in their own skill directories still need the bundled skill directory copied or linked into that location.

Build and install the extension

Install dependencies and build the unpacked extension:

pnpm install
pnpm build

In the daily Chromium-based browser profile:

  1. Open chrome://extensions (or the browser's equivalent extensions page).
  2. Enable Developer mode.
  3. Choose Load unpacked.
  4. Select this repository's .output/chrome-mv3 directory.

wxt.config.ts contains one stable public manifest.key for the local Developer Mode build. It keeps the extension ID stable across builds. The private key is not part of this repository; changing or removing the checked-in public key changes the ID and requires reinstalling the unpacked extension.

The extension only accepts activation messages from numeric loopback pages: http://127.0.0.1/*. It does not connect to a Relay while idle.

Publish the Python package

The published distribution is octoparse-relay. Its release version is the [project].version value in pyproject.toml. Nexus release versions are immutable: increase that version before publishing any changed package.

Publish only to https://nexus.bazhuayu.com/repository/pypi-hosted/. The nexus alias used below must target this endpoint. Do not publish to pypi-group or to any /simple/ endpoint.

Build, validate, and upload both distributions:

release_dir="$(mktemp -d)"
uv build --no-sources --out-dir "$release_dir"
uvx twine check "$release_dir"/*.whl "$release_dir"/*.tar.gz

uvx twine upload --non-interactive -r nexus \
  "$release_dir"/*.whl \
  "$release_dir"/*.tar.gz

Run the Playwright example

Install Python and dependencies with uv:

uv python install 3.11
uv sync --all-groups

Run the example from the repository root:

uv run python examples/relay_playwright.py

When no paired CLI Relay is available, each run performs this sequence:

playwright_relay.connect()
  → bind a fresh 127.0.0.1 port
  → issue a fresh session token
  → open the local /activate URL in the system default browser
  → wake the installed extension through externally_connectable
  → wait for the extension WebSocket handshake
  → connect Playwright through the Octoparse Relay CDP endpoint

The example then opens https://example.com and prints Example Domain. In this temporary mode, closing the returned Playwright browser closes the Octoparse Relay and sends the extension back to its idle state. A later playwright_relay.connect() creates a new port and token; no previous Octoparse Relay URL or Profile path is saved.

Relay always uses the native browser-window viewport. It ignores Playwright viewport overrides, including later calls that try to resize a page.

Relay intentionally supports top-level page targets and iframe DOM frames, including out-of-process iframes. It does not expose worker, shared-worker, or service-worker targets through its CDP bridge.

The default browser must be a Chromium-based browser with this unpacked extension installed in the Profile that receives externally opened URLs. The SDK does not scan or persist browser Profiles. If the extension is missing, disabled, or the URL opens in another Profile, playwright_relay.connect() times out waiting for the authoritative extension handshake.

Development browser

pnpm dev remains useful for WXT extension development and opens a separate development browser Profile. The Developer Mode distribution test above is the production-shaped path: build first, load .output/chrome-mv3 into the daily Profile, then run the Python example.

Relay connection protocol

Both peers use /relay on the same loopback port. The extension connects with role=extension; Playwright connects with role=client. Both URLs carry the same fresh session_token in the token query parameter. Temporary Relays accept one connection per role. CLI Relays keep one extension connection and accept sequential CDP clients, resetting each client's debugger state while retaining its physical pages. The token authorizes the session; roles select the peer protocol and do not provide separate credential permissions. Update the SDK and extension together.

The client sends native CDP requests directly. Extension control messages use type as their message discriminator; the extension handshake uses protocol for its version.

Metadata

Release files for octoparse-relay 0.1.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for octoparse-relay 0.1.10
File Size Uploaded Provenance Actions
octoparse_relay-0.1.10.tar.gz 395.7 kB Without provenance Details for octoparse_relay-0.1.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for octoparse-relay 0.1.10
File Interpreter ABI Platform Actions
octoparse_relay-0.1.10-py3-none-any.whl Python 3 none any Details for octoparse_relay-0.1.10-py3-none-any.whl

Total release size: 426.3 kB

Release files / octoparse_relay-0.1.10.tar.gz

Download URL octoparse_relay-0.1.10.tar.gz
Size 395.7 kB
Tags Source
SHA-256 checksum
How to use checksums
3aef4183e531482cab075e053c1b1eee384d371a22c094145cece76b8fbb9877
BLAKE2b-256 checksum
How to use checksums
4116c4baff6a9a12869a41148cb0e1631e0eb82eefd9aaa626a48a8596b58cae
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / octoparse_relay-0.1.10-py3-none-any.whl

Download URL octoparse_relay-0.1.10-py3-none-any.whl
Size 30.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
cb0066925d5fc5098bd29751eff53e8b4fd4e232c0e5ef654d8a9fb841220f18
BLAKE2b-256 checksum
How to use checksums
e3b7e3f6a34117645d72f404262abda1bb40c16a5f64f164332e0d556ab779ef
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for octoparse-relay 0.1.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.10 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page