Try it

Python 3.12 or newer, on Linux, macOS, or Windows through WSL (each release is installed from PyPI on fresh Ubuntu 24.04 and macOS machines and checked):

pip install "frankensurf[mcp]"
frankensurf install      # every free tool: browsers, stealth browsers, local search, Crawl4AI
frankensurf read "https://www.walmart.com/search?q=air+fryer" --explain

Like playwright install: frankensurf install search chromium installs just those, --with-deps adds the browsers’ system libraries on Linux (it asks for sudo), frankensurf install --list shows what is installed where, and frankensurf uninstall all removes it again. No Docker needed. (Crawl4AI is Linux and WSL only for now; on macOS a plain install skips it.)

No account or key needed. The read climbs until it gets the real page:

https://www.walmart.com/search?q=air+fryer
  ✗ http                   CAPTCHA                0.4s
  ✗ local                  CAPTCHA                1.5s
  ✓ camoufox               got the page           15.6s

air fryer - Walmart.com
24,192 characters via camoufox, complete (107 results), free

Drop --explain for the full JSON: text or markdown, JSON-LD, the page’s embedded and fetched JSON, images, and the receipt.

Free or paid

Everything works with no keys. Paid tools are opt-in, and only come in once a site has walled a free tool, cheapest likely success first. From the newest benchmark below (150 sites picked before any was read, read once):

With no paid tools at all, it reads as many sites as the best paid tool on its own (88.0% against Zyte’s 86.0% and Firecrawl’s 84.7%; at 150 sites that gap is within the noise).

To allow paid tools, set their keys (for example FIRECRAWL_API_KEY) and pass --paid (--max-cost USD caps a read; from Python, allow_paid_fallbacks and max_cost_usd). See Paid tools and budgets.

Search

search uses free sources first and checks every result for topic, kind of page and date; Exa and Firecrawl’s search join only when you allow paid tools. Judged blind (results pooled and shuffled, which tool found them hidden), useful results in the top five:

Free, it beats Firecrawl’s search (blind set 6, the 69 queries Firecrawl answered: 3.78 against 2.91, better on 35 queries and worse on 5). With paid allowed it is level with Exa. Without the local search engine (frankensurf install search), free search is much weaker. Details: benchmarks/2026-10-13-search.

Use it

From an agent, as an MCP server (setup for each client):

claude mcp add frankensurf -- frankensurf-mcp

From Python:

from frankensurf import Runtime

async with Runtime("state") as web:
    page = await web.read("https://example.com", policy_overrides={"prefer_markdown": True})
    hits = await web.search("playwright python tutorial")

From the command line:

frankensurf read https://example.com --markdown
frankensurf search playwright python tutorial
frankensurf watch https://news.ycombinator.com/ --link-pattern 'item\?id=\d+'

What you get back is raw page data, not answers. Your agent decides what the page means, and if it isn’t what it wanted, it can ask again with retry_of=<trace_id> to skip every tool already tried.

Also in the box: profiles (log in once, reuse the session from any tool), human handoff for CAPTCHAs and 2FA, site modules (save what your agent learns about a site as data), and Web Bot Auth request signing.

Want the article, not the menus? Pass --main (MCP main_content=True) for main_text (best with pip install "frankensurf[main]").

Search takes --site, --exclude-domain, --recency and --region on every source, a --vertical (news, reference, discussions, qa, code, papers, books) backed by official free APIs, and --mode merge to fuse every source at once. Running on a server that reads URLs it didn’t choose? Set FRANKENSURF_BLOCK_PRIVATE_NETWORK=1.

Every page read carries receipt.quality: good, partial or poor, with flags such as paywall, menus or placeholder, so an agent (or a paid fallback) knows when a page didn’t really come through.

Slow page? A read with no page after 3 seconds starts two more free tools beside it and keeps the first complete page (--lightning races from the start). A URL that answers not found is looked up with the site’s own search, and receipt.rescued says so. Still walled? Allow a real Chrome on your machine (--real-browser, with its own profile, never yours) or a second connection (FRANKENSURF_EGRESS; frankensurf egress check shows where each comes out).

Keeping up with many sites? frankensurf watch-sites URL… --since DATE lists their new pages from the feeds and sitemaps they publish, polled with conditional requests. Running on a server? Run Steel beside it (docker compose up -d) and, if sites still refuse your data-centre address, set FRANKENSURF_PROXY to a proxy you bring.

Search works without any setup (DuckDuckGo, Bing and the topic sources), and is stronger with a local SearXNG, which answered about a third of the queries in our blind benchmark. frankensurf install search installs one without Docker (SearXNG at a pinned, checksum-checked commit, in its own environment, on 127.0.0.1:8088); search starts it again after a reboot, and frankensurf search-engine start|stop|status manages it. When a search ran without it, receipt.search_routing.local_search says so and how to install it.

Agents learn the tools from a bundled skill: the MCP server sends it as its instructions, and frankensurf skill install adds it to ~/.claude/skills (frankensurf skill uninstall removes it). frankensurf setup, the guided install, asks before installing it and never does in scripts or CI; --install-skill says yes in advance.

Reading many pages from one site? Let Frankensurf learn it first:

frankensurf module discover https://www.example.com/ --query "desk lamp" --save

That finds the site’s search (its schema.org SearchAction, its search form, or by typing once into its search box), reads the results, and saves a site module: where the items live in the page’s own data, plus a search template. From then on every search on that site comes back as structured items, and batch_template runs many searches as one polite batch. Check the sample rows before saving; the draft can pick the wrong list.

How it works

Every way to fetch a page is a rung: plain HTTP and markdown negotiation, browsers, stealth browsers and signed requests, hosted browsers, paid unblockers, and finally a person clearing the wall in a visible browser. A read starts at the cheapest rung and climbs only when a page pushes back. Every tool is a plugin, so new ones slot in as another rung. See How escalation works and the full list of integrations.

Benchmark

scripts/toolbench.py runs each tool on its own, then Frankensurf, on the same pages with the same content checks. The newest: 150 sites in 15 categories picked before any was read and read once (held-out set 7, 11 October 2026, v0.35), raw rows in benchmarks/2026-10-11/:

Frankensurf with paid fallbacks read 17 sites Firecrawl missed; Firecrawl read none it missed. Of the 146 sites any tool read, it got 98.6%. Its median read is faster than Firecrawl’s, but its slowest reads are slower: p90 25.7 s against 10.6 s, when a read climbs several tools. Medians are over pages read; cost is per 1,000 pages read right. ZenRows is left out (its plan ran out of credits), as is Scrapfly. From a fresh cloud VM, a data-centre address sites wall more often: free tools 82.0%, lightning mode 84.0% at a 1.2 s median.

The set before it (set 6, 10 October, every tool alone the same way): 95.3% with paid fallbacks against Firecrawl’s 85.3% and Zyte’s 86.0%. Earlier fresh sets: 152 sites on 6 October (v0.16) 89.5% against Firecrawl’s 80.3%; 139 sites on 9 October (v0.26) 88.5% against 74.1%. See Benchmarks.

Known walls

It’s an alpha, and some sites still win:

• Need the paid tools: in a free-only spot check on 9 October, Etsy, Home Depot, Yelp, Crunchbase and G2 blocked every free tool.

• Beat everything so far: Shopee, Temu and Idealista.

• Social sites without a login: LinkedIn, X, YouTube, Threads, Bluesky and Reddit read fine. Instagram, TikTok, Facebook and Pinterest show only the public shell (name, bio, counts); the posts need a login.

• Logged-in pages (your feeds, your orders) need a profile.

• A slow tail: most reads take a few seconds, but one in ten climbs several tools and takes 25 s or more.

Found a site it can’t read? Report it; every report becomes a test case.

Develop

From a clone:

python3 -m venv .venv
.venv/bin/pip install -e '.[test,mcp]'
.venv/bin/frankensurf install
PYTHONPATH=.:src .venv/bin/pytest -q

The product design is in SPEC.rst; contributor rules are in AGENTS.md. New tools are plugins, never branches in Core; see Writing a plugin for building, installing and trusting your own (frankensurf plugins), with a working example in examples/frankensurf-plugin-example. The website and docs live in site/.

Licence

Apache-2.0. See LICENSE.