Skip to main content

MCP server for scraping Google AI Overviews via a Chrome extension relay

Project description

google-ai-scraper

MCP server for scraping Google AI Overviews via a Chrome extension relay. No API keys needed. It uses your own browser session and a shared local FastAPI backend.

How it works

MCP Client (Claude Code, Cursor, etc.)
    ↕ stdio or SSE
MCP Server (this package)
    ↕ HTTP
Shared FastAPI Backend (:15551)
    ↕ polling
Chrome Extension
    → opens background tab on google.com
    → scrapes AI Overview → Markdown
    → posts result back

In default stdio mode, each MCP client process auto-reuses or auto-starts the shared backend on 127.0.0.1:15551.

Install

# Via uvx (recommended)
uvx google-ai-scraper

# Or pip
pip install google-ai-scraper

Prerequisites

  1. Chrome with the Google AI Overview Scraper extension installed
  2. Python 3.10+
  3. Extension options pointed at http://localhost:15551

MCP Client Configuration

Claude Code

{
  "mcpServers": {
    "google-ai-scraper": {
      "command": "uvx",
      "args": ["google-ai-scraper"]
    }
  }
}

Claude Desktop

{
  "mcpServers": {
    "google-ai-scraper": {
      "command": "uvx",
      "args": ["google-ai-scraper"]
    }
  }
}

Cursor

{
  "mcpServers": {
    "google-ai-scraper": {
      "command": "uvx",
      "args": ["google-ai-scraper"]
    }
  }
}

Local Source Checkout

{
  "mcpServers": {
    "google-ai-scraper": {
      "command": "uv",
      "args": [
        "run",
        "--directory",
        "/absolute/path/to/server",
        "google-ai-scraper"
      ]
    }
  }
}

MCP Tools

Tool Params Description
search query Search Google AI Overview. Returns markdown + citations + thread_id
follow_up query, thread_id Continue a conversation in an existing thread
health Check server + extension connectivity

CLI Options

google-ai-scraper [--sse] [--no-server] [--backend] [--port PORT]
Flag Default Description
--sse off Run the MCP server with SSE transport instead of stdio
--no-server off Do not auto-start or reuse the shared FastAPI backend
--backend off Run only the shared FastAPI backend
--port 15551 FastAPI backend port

Environment Variables

Variable Default Description
GOOGLE_AI_SCRAPER_URL http://127.0.0.1:15551 FastAPI backend URL

Advanced Modes

Explicit Shared Backend

uv run google-ai-scraper --backend --port 15551

SSE Server

uv run google-ai-scraper --sse --no-server

Shared Backend + SSE Helper Script

cd server
./start-services.sh

This starts the shared backend on :15551 and the SSE endpoint on :8001.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

google_ai_scraper-0.2.11.tar.gz (52.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

google_ai_scraper-0.2.11-py3-none-any.whl (10.0 kB view details)

Uploaded Python 3

File details

Details for the file google_ai_scraper-0.2.11.tar.gz.

File metadata

  • Download URL: google_ai_scraper-0.2.11.tar.gz
  • Upload date:
  • Size: 52.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.1 {"installer":{"name":"uv","version":"0.11.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for google_ai_scraper-0.2.11.tar.gz
Algorithm Hash digest
SHA256 7c4e253e5b390079a689c13bbe4f238c6ecef3e4362d11624a80bd4e74c63fb1
MD5 73e175e5ed6abaef8ab8e62363f58eab
BLAKE2b-256 8e1026e105a1e4c7e6b1d1ac905743897fef63f7b6583099b15df12344bcf79e

See more details on using hashes here.

File details

Details for the file google_ai_scraper-0.2.11-py3-none-any.whl.

File metadata

  • Download URL: google_ai_scraper-0.2.11-py3-none-any.whl
  • Upload date:
  • Size: 10.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.1 {"installer":{"name":"uv","version":"0.11.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for google_ai_scraper-0.2.11-py3-none-any.whl
Algorithm Hash digest
SHA256 999a0218478116626412d6058fbf415c7016157daacf61cd7a41ba0fbd73181e
MD5 3a6d2d7a4bf93650a0a3af1ec70b384d
BLAKE2b-256 0d19a9234b54ea37f218d09b8df9172cb5a2bed5f7dca476d063ec88908a1774

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page