voidcrawl 0.2.3

Rust-native CDP browser automation for Python via PyO3

VoidCrawl

CDP browser automation for Python — a Rust-native Chrome DevTools Protocol client exposed to Python via PyO3.

void_crawl replaces Playwright/Selenium with a permissively-licensed (Apache-2.0) stack for rendering JavaScript-heavy pages. Built on chromiumoxide with a shared Tokio runtime.

Used by Yosoi — an AI-powered selector discovery tool for resilient web scraping.

Requirements

• Rust ≥ 1.86 (edition 2024)
• Python ≥ 3.10
• Chrome/Chromium installed on the system
• maturin ≥ 1.7 (cargo install maturin)

Installation

# Build and install into your venv
./build.sh

# Or manually:
maturin develop --release --manifest-path crates/pyo3_bindings/Cargo.toml

Quick Start

BrowserPool (recommended)

Tabs are recycled, not closed — near-instant reuse across requests.

import asyncio
from void_crawl import BrowserPool

async def main():
    async with BrowserPool.from_env() as pool:
        async with pool.acquire() as tab:
            await tab.navigate("https://example.com")
            print(await tab.title())
            print(len(await tab.content()))

asyncio.run(main())

BrowserSession (low-level)

import asyncio
from void_crawl import BrowserSession

async def main():
    async with BrowserSession(headless=True) as browser:
        page = await browser.new_page("https://example.com")
        print(await page.title())
        await page.close()

asyncio.run(main())

Docker

Pre-built multi-arch images (linux/amd64, linux/arm64) are published to GHCR on every push to main and every tagged release:

docker run -d --network host --shm-size=2g \
  ghcr.io/cascadinglabs/voidcrawl:headless-latest

export CHROME_WS_URLS="http://localhost:9222,http://localhost:9223"
python examples/basic_navigation.py

Or via compose:

docker compose -f docker/docker-compose.yml up -d

Available tags: headless-latest, headless-<version>, headless-<sha>, and the same set prefixed headful- for GPU + VNC (linux/amd64 only). See the Docker & VNC guide and the Docker Config reference for every runtime knob.

Testing

# Rust integration tests (serial — Chrome singleton lock)
cargo test -p void_crawl_core -- --test-threads=1

# Python integration tests (requires built extension + Chrome)
uv run pytest tests/ -v

Documentation

• Full API reference
• Examples

License

Apache-2.0