dlbrowser 4.0.0

Enterprise-Grade Multi-Backend Browser Engine for AI Agents and Web Automation

DLBrowser — Reliable, metered web access for AI agents

Give your agent eyes. One MCP connection, auto-recovers through Cloudflare blocks, captchas, and JS-heavy pages that break every other tool. Every request metered against an API-key plan — transparent credits, no surprises.

# One line. Works everywhere.
npx dlbrowser connect --key dlb_xxx
pip install dlbrowser
dlbrowser fetch https://example.com

---

Why DLBrowser

Every AI agent eventually hits the wall of the live web.

Cloudflare, DataDome, and PerimeterX block headless browsers. Captchas stop naive fetches. One site works with curl, another needs a real browser, another needs residential proxies. You have no idea which backend will work until you try — and no record of what each run costs.

DLBrowser solves this with a self-healing engine that tries the fastest backend first, falls back through increasingly stealthy alternatives, and logs every outcome into a queryable performance database that makes the router smarter over time.

It is not a scraper. It is not a crawler. It is an acquisition infrastructure platform — the capability layer that gives AI agents reliable, observable, and billable access to any web resource.

---

Quick Start

For AI Agents (one line)

# Add to Claude Code
claude mcp add dlbrowser --transport http \
  --url https://api.dlbrowser.dev \
  --header "Authorization: Bearer dlb_xxx"

# Or use the universal installer
npx dlbrowser connect --key dlb_xxx

# Now your agent has 11 tools:
# dlbrowser_fetch · dlbrowser_scrape · dlbrowser_smart_extract
# dlbrowser_screenshot · dlbrowser_markdown · dlbrowser_crawl
# dlbrowser_search · dlbrowser_serp · dlbrowser_status
# dlbrowser_account · dlbrowser_topup

For Developers

# Install
pip install "dlbrowser[full]"
python3 -m playwright install chromium

# One-shot fetch
dlbrowser fetch https://news.ycombinator.com

# Check your usage
dlbrowser usage --user usr_abc --period monthly

# Start the MCP server
dlbrowser mcp --transport http --port 8929

For Chrome Users

Install the extension from the Chrome Web Store (Cmd+Shift+D to scrape any page) or load it unpacked from extension/.

---

How It Works

Agent / CLI / Extension / MCP Client
          │
    ┌─────┴─────┐
    │  MCP Gateway │  ← Paywalled (401/402/403)
    │  11 tools    │  ← free: status, account, topup
    └─────┬─────┘
          │
    ┌─────┴─────┐
    │   Router   │  ← Adaptive scoring: success×0.4 + latency×0.2 + cost×0.2 + freshness×0.2
    └─────┬─────┘
          │
    ┌─────┴──────────────────────────────┐
    │  curl_cffi → tls_client            │  ← HTTP-level, 50ms, no JS
    │  → playwright → nodriver → firecrawl│  ← Browser-level, 1.5-2.5s, full JS
    └─────┬──────────────────────────────┘
          │
    ┌─────┴─────┐
    │  Ledger   │  ← per-run cost, margin, trace_id
    └─────┬─────┘
          │
    ┌─────┴─────┐
    │  Invoice  │  ← credits × unit price
    └───────────┘

Every operation produces a run_id, trace_id (OpenTelemetry-compatible), backend, credits_consumed, and estimated_cost. The router learns per-domain — more usage means better routing.

---

Backends

Backend	Speed	Stealth	JS	Best For
curl_cffi	50ms	Medium	No	Simple sites, APIs — 30x faster than browser
tls_client	200ms	Medium	No	TLS fingerprint bypass
undetected-playwright	1.5s	High	Yes	JS-heavy sites, SPAs
nodriver (CDP)	2.5s	Highest	Yes	Aggressive anti-bot, direct CDP
Firecrawl	varies	Paid	Yes	Last resort (pay per-use)

---

Features

Self-healing fallback	5 backends auto-escalate on block. Never silently fails.
Adaptive routing	Per-domain scoring. Learns which backends work for which sites.
SPA detection	Detects Next.js/React shells and auto-escalates to a real browser.
Atomic quota	0% billing overshoot. Reservation-based — no surprise bills.
Per-run cost tracking	Every request records infrastructure cost and gross margin.
Captcha solving	3 providers (2captcha, anti-captcha, capsolver) for hCaptcha/reCAPTCHA/Turnstile.
LLM extraction	Natural language → structured JSON. "Extract product prices and ratings."
Deep crawling	BFS/DFS/Best-First with crash recovery and robots.txt compliance.
Markdown conversion	Full HTML→Markdown with tables, code blocks, mermaid diagrams.
OpenTelemetry tracing	Every run is traceable end-to-end.
Open source (MIT)	Self-host with full feature parity. Your data never leaves your infra.
Framework adapters	LangChain, LlamaIndex — drop-in tools for any agent framework.

---

Pricing

Plan	Price	Credits/mo	Best for
Scout	Free	2,500 (one-time)	Evaluation, hobby projects
Analyst	$29/mo	100,000	Indie devs, small teams
Operator	$85/mo	500,000 (+ captcha)	Production workloads
Command	$299/mo	2,000,000 (+ proxy)	Scale teams
Enterprise	Custom	Unlimited	Dedicated infra, SLA, support

Overage: $0.001/credit. Annual: 20% off. No credit multipliers — what you see is what you pay.

---

Enterprise

• Multi-tenant — Organizations, teams, seats, roles
• API keys — Create, revoke, rotate, scope by permission, set expiry
• Audit logs — Immutable event ledger: who did what, when, at what cost
• Usage dashboard — Credits, cost, success rate, top domains
• Webhooks — HMAC-SHA256 signed events with dead-letter queue
• SLA monitoring — Scheduled page diffing with noise-tolerant change detection
• Self-hosted — Docker one-liner, full feature parity

---

When NOT to Use DLBrowser

• You need pre-built datasets → use Bright Data/Apify
• You need a visual scraping IDE → use Browserflow/Octoparse
• You need screenshot-as-a-service → use Urlbox
• You need an enterprise proxy network → use Bright Data

DLBrowser is for teams running AI agents that need reliable, observable, cost-controlled web access at scale.

---

Benchmarks

[Coming soon — running the full test suite against Firecrawl, Crawl4AI, and naive fetch on 50 production sites]

The codebase supports 5-backend fallback, adaptive routing, and per-run cost tracking that no competitor has. We'll publish benchmark data once the managed gateway is live.

---

Documentation

Resource	Link
Getting Started	dlbrowser.dev/getting-started
API Reference	docs.dlbrowser.dev
Architecture	docs.dlbrowser.dev/architecture
Self-Hosted Deploy	DEPLOYMENT.md
Comparison	dlbrowser.dev/comparison
Enterprise	dlbrowser.dev/enterprise
MCP Setup	docs.dlbrowser.dev/mcp

---

The Team

Built by Dekrypt Labs. Contact: [e-mail]

License

MIT — Dekrypt Labs