parsek-cdp-server

The server side of parsek-cdp: launching, supervising and proxying a browser, plus the server-side feature producers that digest raw CDP into Parsek.* events.

A separate distribution from parsek-cdp (the client); installed via pip install parsek-cdp[server]. It depends on parsek-cdp and reuses its shared layers wholesale — the generated cdp protocol, the core primitives (CDPConnection, Target, DataType) and the parsek contract — adding only what must live next to the browser.

Документация на русском: README.ru.md.

Running

import asyncio
from parsek_cdp_server import ParsekServer
from parsek_cdp.features import RequestListener


async def main():
    server = ParsekServer(idle_timeout=300,    # a browser self-closes after 5 min idle
                          inactivity_timeout=600)  # ...or 10 min without a client message
    server.register_feature(RequestListener)   # make the feature selectable via ?feature=
    await server.start("127.0.0.1", 9333)
    try:
        await asyncio.Event().wait()           # keep the server alive
    finally:
        await server.stop()


asyncio.run(main())

A client connects to this server with Browser.get_distant_browser("http://127.0.0.1:9333", ...) — the API is identical to a local browser (see the client README). Every POST /browsers spins up its own supervised browser. Two timeouts let a browser close itself, each settable on the server or per-request in the body (omit, or pass a non-positive value, to disable):

• idle_timeout — seconds with no connection at all;
• inactivity_timeout — seconds without a single client message, connected or not, which is what catches a client that connected and then went quiet. Only what the client sends counts, so keep it above the longest the client may legitimately spend waiting on the browser (a slow navigation, a wait_for) — waiting sends nothing and the clock keeps running.

Either way the client is told why: the control channel gets a Parsek.browserStateChanged with state=closed and the reason before the browser goes down.

Where the browser comes from

launcher finds a Chromium-compatible binary on its own. By default it walks the usual install paths; the PARSEK_CHROMES_PATH env var (directories separated by os.pathsep, like PATH) overrides the search — each launch picks a random one of the browsers it finds (a cheap way to spread load and fingerprints). An explicit path and flags go through LaunchOptions (executable, headless, extra_args, ...) — either on the server or per-request in the POST /browsers body.

Modules

Connection model

As in the client, every target gets its own websocket (no sessionId multiplexing). One parsing flow = one browser_context; pages within it each connect over their own websocket. The proxy bridges a page's client socket to the corresponding Chrome socket 1:1.

Endpoints

HTTP POST /browsers                                   create a task -> browser_uuid
HTTP GET  /metrics                                    Prometheus metrics
ws        /cdp/{browser_uuid}/control                 browser-level pipe + Parsek.browserStateChanged
ws        /cdp/{browser_uuid}/page/{target_id}        CDP proxy (per-page) + Parsek.*

Responses

POST /browsers returns 200 application/json — the browser id and its control-channel endpoint:

{
  "browserUuid": "1f2ab34cd56e78f9...",
  "wsUrl": "ws://127.0.0.1:9333/cdp/1f2ab34cd56e78f9.../control"
}

GET /metrics returns 200 text/plain in the Prometheus text exposition format (see the metrics table below).

The websocket endpoints (/control and /page/{target_id}) respond with a websocket upgrade; for an unknown browser_uuid the socket is closed immediately with code 4040 ("unknown browser").

The control channel is a raw pipe to the browser endpoint plus the Parsek.browserStateChanged broadcast, so a parser learns when its browser crashed/restarted instead of hanging; browser-level CDP (creating/disposing contexts, etc.) passes straight through it.

Features

The server is pure passthrough by default (ParsekServer has no features argument). Feature producers are added with register_feature(...) and selected per browser with POST /browsers?feature=<name>. On a page pipe a producer digests the raw CDP burst into a couple of aggregated Parsek.* events and suppresses the raw events it consumed; ?raw=1 opts out of suppression.

Metrics (GET /metrics)

Targets and CPU/RAM are sampled by a background task (polling /json/list + psutil), so a scrape only reads cached values and does not block the event loop. The event counter is labelled by target id — series for dead targets are pruned on every snapshot.

Zombie reaper

On start the server spawns a separate subprocess (reaper) that periodically scans for Chrome processes we launched (recognised by the --user-data-dir=…parsek-cdp-… marker) that became zombies or orphans (ppid == 1), and kills them along with their subtree. Controlled by ParsekServer(reap_zombies=..., reap_interval=...); it can also be run by hand: python -m parsek_cdp_server.reaper --once.

Notes

The Parsek.* contract lives in the client distribution (parsek_cdp.parsek) and is shared by both sides.

License

Apache-2.0.