turbo-surf (Python)
Browserless, native-speed web crawler + extractor with a real V8 JS-render tier — a PyO3 binding over the turbo-surf Rust engine. Fetch-free: you pass a page's HTML in, and get a view out (Markdown, visible text, links, a typed extraction, an accessibility tree). For JS-gated pages, the engine runs the page's own scripts in a true V8 isolate over a native DOM — no headless Chromium.
import turbo_surf as ts
html = open("page.html").read()
ts.markdown(html, base_url="https://example.com/") # -> Markdown str
ts.text(html) # -> visible text
ts.links(html, base_url="https://example.com/") # -> list[str]
# Typed extraction: a JSON schema maps field names to selector specs.
schema = '{"title": {"selector": "h1"}, "prices": {"selector": ".price", "list": true}}'
ts.extract(html, schema, base_url="https://example.com/") # -> JSON str
# JS-gated page: run its own scripts, read the hydrated DOM.
hydrated = ts.render(html, script, base_url="https://example.com/")
Fatal faults (malformed schema JSON, a render-tier failure) raise
turbo_surf.TurboSurfError; the non-JS views never raise.
Install
pip install turbo-surf
Prebuilt abi3 wheels (CPython 3.8+) are published for Linux (x86_64/aarch64), macOS (arm64), and Windows (x64).
API
| function | returns | notes |
|---|---|---|
markdown(html, base_url="") |
str |
Markdown render |
text(html) |
str |
visible text |
title(html) |
str |
document <title> |
html(html) |
str |
re-serialized HTML |
links(html, base_url="") |
list[str] |
resolved hyperlink targets |
interactive_elements(html, base_url="") |
JSON str |
links/buttons/inputs |
accessibility_tree(html) |
JSON str |
a11y tree |
hydration_state(html) |
JSON str |
hydration probe |
detect(html) |
JSON str |
is the page JS-gated? |
query(html, selector, kind=None) |
JSON str |
kind = "css"/"xpath"/auto |
extract(html, schema_json, base_url="") |
JSON str |
typed extraction |
evaluate(html, script) |
str |
run script over the DOM (sync) |
render(html, script, base_url="") |
str |
hydrated HTML after page scripts run |
transform(src, ts=False, jsx=False) |
str |
TS/JSX → classic JS (swc) |
MIT licensed.
1 maintainer
Metadata
1 maintainer
Release files for turbo-surf 0.5.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| turbo_surf-0.5.3.tar.gz | 733.2 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| turbo_surf-0.5.3-cp38-abi3-win_amd64.whl | CPython 3.8 | abi3 | Windows x86-64 | Details |
| turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.8 | abi3 | Linux glibc 2.17+ x86-64 | Details |
| turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | CPython 3.8 | abi3 | Linux glibc 2.17+ ARM64 | Details |
| turbo_surf-0.5.3-cp38-abi3-macosx_11_0_arm64.whl | CPython 3.8 | abi3 | macOS 11.0+ ARM64 | Details |
Total release size: 122.5 MB
Release files / turbo_surf-0.5.3.tar.gz
| Download URL | turbo_surf-0.5.3.tar.gz |
|---|---|
| Size | 733.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
05e8bad8e0a5c10fa7e3767d41e27b47388a24b213a679ded99c29536e8b2c8a
|
|
BLAKE2b-256 checksum How to use checksums |
c497a5b69acf2f1b4e9b705f30ec01eef0d157db29856450732c0b403c3cc460
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / turbo_surf-0.5.3-cp38-abi3-win_amd64.whl
| Download URL | turbo_surf-0.5.3-cp38-abi3-win_amd64.whl |
|---|---|
| Size | 28.2 MB |
| Tags | CPython 3.8 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
d1552653f64fe0c6c912a595dbefa200e47b4d754b18e058848e327d40aa2ad5
|
|
BLAKE2b-256 checksum How to use checksums |
9123e2c59d5d3043d5cf8006c008898bac2c1adb1b5249ea260c6adb0ffcbc6f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 31.4 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
5a2592e090ae26be8b4eebae32a7be40c2a76125ce7e7ef51acc49015649152b
|
|
BLAKE2b-256 checksum How to use checksums |
0b99d10db410d91a8d57eee653a3b4a46bd5cbd0a7572e86b99bc6b69c2e794e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | turbo_surf-0.5.3-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 33.1 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
ba8b4be4dd346eeada4033615cadb789b5193b40491cd0b3fb6cc59482764172
|
|
BLAKE2b-256 checksum How to use checksums |
80c84249b29ee747839f882d32d3c8ce6f4a25c2cd0ae9adcb334838db8fac38
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / turbo_surf-0.5.3-cp38-abi3-macosx_11_0_arm64.whl
| Download URL | turbo_surf-0.5.3-cp38-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 28.9 MB |
| Tags | CPython 3.8 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ac70ef767b570590e6459f6b37ca90f6efc5a79c063f4b21b006cc79b865bcfd
|
|
BLAKE2b-256 checksum How to use checksums |
590d1e9d7789ab6530fb79c2d8f77ccbef1f00e8e4473d0fd32d6803f6849a14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
