Wafle-Scraper — Universal MCP Server for Safe Web Scraping

Wafle-Scraper is an MCP server that lets AI agents (OpenCode, Claude, Cursor) extract data from the web safely and responsibly.

Safety First

Quick Start

pip install wafle-scraper
playwright install chromium
wafle-scraper

Configuration

OpenCode / Claude Desktop / Cursor

{
  "mcpServers": {
    "wafle-scraper": {
      "command": "wafle-scraper",
      "description": "Web scraping & browser automation — incognito, audited, safe"
    }
  }
}

CLI Options

wafle-scraper                    # MCP stdio mode (default for agents)
wafle-scraper --http --port 8000 # HTTP SSE mode
wafle-scraper --version          # Show version

MCP Tools

Browser Backend (Playwright)

• Incognito always: storage_state=None, fresh context per session
• No permissions: No camera, mic, location access
• Anti-detection: Rotating UA, viewport, locale, timezone
• Natural delays: Human-like timing between actions
• Gradual scroll: Loads lazy content naturally

CAPTCHA Handling

Wafle-Scraper does NOT solve CAPTCHAs automatically. When a CAPTCHA is detected:

1. The scraper pauses
2. Prompts you to open the URL in your browser
3. You solve the CAPTCHA manually
4. Type done and the scraper continues

This is the only ethical and reliable approach without paid services.

Security

• Blocked: localhost, 127.0.0.1, private IPs, file://, chrome://
• Rate limiting (configurable, default 2s min interval)
• Scope enforcement — only processes what you explicitly request
• User-Agent rotation
• Browser isolation — Playwright contexts are fully sandboxed

Requirements

• Python 3.10+
• Playwright with Chromium installed (playwright install chromium)
• Windows, macOS, Linux

Installation from Source

git clone https://github.com/creandoaldia/wafle-scraper.git
cd wafle-scraper
pip install -e .
playwright install chromium

License

MIT

Why "Wafle-Scraper"?

Part of the WAFLE ecosystem (Web AI Framework for Language Ecosystems). Wafle-Scraper gives WAFLE agents the ability to read the live web — safely, transparently, and under your control.