voidcrawl 0.1.9

Rust-native CDP browser automation for Python via PyO3

[!WARNING] VoidCrawl is currently in Alpha. The API is expected to change significantly. We do not expect a stable API until we are out of Beta.

VoidCrawl

CDP browser automation for Python — a Rust-native Chrome DevTools Protocol client exposed to Python via PyO3.

void_crawl replaces Playwright/Selenium with a permissively-licensed (Apache-2.0) stack for rendering JavaScript-heavy pages. Built on chromiumoxide with a shared Tokio runtime.

Used by Yosoi — an AI-powered selector discovery tool for resilient web scraping.

Requirements

• Rust ≥ 1.86 (edition 2024)
• Python ≥ 3.10
• Chrome/Chromium installed on the system
• maturin ≥ 1.7 (cargo install maturin)

Installation

# Build and install into your venv
./build.sh

# Or manually:
maturin develop --release --manifest-path crates/pyo3_bindings/Cargo.toml

Quick Start

BrowserPool (recommended)

Tabs are recycled, not closed — near-instant reuse across requests.

import asyncio
from void_crawl import BrowserPool

async def main():
    async with BrowserPool.from_env() as pool:
        async with pool.acquire() as tab:
            await tab.navigate("https://example.com")
            print(await tab.title())
            print(len(await tab.content()))

asyncio.run(main())

BrowserSession (low-level)

import asyncio
from void_crawl import BrowserSession

async def main():
    async with BrowserSession(headless=True) as browser:
        page = await browser.new_page("https://example.com")
        print(await page.title())
        await page.close()

asyncio.run(main())

Docker

docker compose -f docker/docker-compose.yml up -d

export CHROME_WS_URLS="http://localhost:9222,http://localhost:9223"
python examples/basic_navigation.py

For headful Docker with GPU + VNC, see docs/docker-headful.md.

Testing

# Rust integration tests (serial — Chrome singleton lock)
cargo test -p void_crawl_core -- --test-threads=1

# Python integration tests (requires built extension + Chrome)
uv run pytest tests/ -v

Documentation

• Full API reference
• Examples

License

Apache-2.0