www-search-mcp

MCP server for web search, HTTP fetch, file download, API requests, and package search (PyPI, GitHub).

Optimized for batching: every tool accepts lists (multi-query / multi-URL) to reduce round-trips.

Quick Install

# Run without installing (recommended)
uvx www-search-mcp

# Or install as a global tool
uv tool install www-search-mcp

VS Code / Cursor config:

{
  "mcpServers": {
    "www-search": {
      "command": "uvx",
      "args": ["www-search-mcp"]
    }
  }
}

Tools Overview

Batching: pass queries=["q1", "q2"] or urls=["url1", "url2"] instead of calling one-by-one. Every tool also accepts output_format ("object" default, "json", "yaml"; web_fetch additionally supports "markdown" — the full cached page text, no metadata wrapper, for re-calling after a truncated fetch).

Search Tools

web_search

queries: str | list[str]
max_results: int = 5

Returns title, url, snippet. Safe search disabled.

web_search_images

queries: str | list[str]
max_results: int = 5

Returns title, image, thumbnail, height, width, source.

web_search_github

queries: str | list[str]
max_results: int = 5

Returns title, url, stars, forks, language. Uses GitHub REST API (rate-limited to ~10/min without token).

web_search_pypi

queries: str | list[str]
max_results: int = 5

Returns name, version, summary, author, license, requires_python, dependencies.

Fetch & Download Tools

web_fetch

urls: str | list[str]      # up to 50 per call
fetch_div: str = ""        # CSS selector (e.g. "article")
save_path: str = ""        # absolute path for full content (single URL)
cache: "always" | "none" | "replace" = "always"
max_chars: int = 25000     # inline text cap per URL; with N URLs split your budget

Returns Markdown per URL in results[]. Per-item cache_key and truncated-flag; when inline text is cut, full content goes to the reported channel (save_path / download_url on HTTP). Binary content is rejected — use web_download for files.

web_download

urls: str | list[str]
save_path: str | list[str]  # required; absolute path(s), one per URL

stdio-only tool (absent in HTTP transport). Downloads files to disk and reports save_path, bytes, mime_type per item; binary content is never returned inline.

API Tool

web_request

queries: dict | list[dict]

Spec fields per query:

• type: "rest" | "graphql"
• method: "GET" | "POST" | ...
• url: target URL
• headers: optional dict
• requests: repeat count (default 1)
• concurrency: async workers (default 1)
• time: duration in seconds (0 = fixed count)

REST: body: dict|list|str GraphQL: query: str, variables: dict, operationName: str

Auth: WEB_REQUEST_TOKEN used as default Authorization if not provided in headers.

Output: status_counts, http_status_counts, latency_ms percentiles. Small runs (<=3) include response samples.

Diagnostic Tools

mcp_info

Server configuration, tool descriptions, environment variables.

mcp_status

Real-time diagnostics:

• uptime_seconds, pid, python_version
• throttle: last request, interval
• sessions: total, stale
• connections: niquests version, pool size
• resources: FD limit, memory RSS, event loop tasks
• counters: requests, errors, timeouts
• config: transport, timeouts
• health: DDGS availability
• metrics: latency percentiles (p50/p95/p99), subtask stats

Privileged status server: A separate Starlette-based HTTP server runs on a daemon thread (default port 8081), providing /status, /healthz, and /tasks endpoints. This server remains responsive even under heavy load because it operates independently from the main event loop. Configure via WEB_STATUS_PORT (set to 0 to disable).

Diagnostic HTTP Routes

When running in HTTP transport mode (WEB_TRANSPORT=streamable-http):

• GET /healthz — Health check
• GET /readyz — Readiness probe (200/503)
• GET /status — Same JSON as web_mcp_status
• GET /tasks — Active event loop tasks (debugging)
• GET /memory — Memory breakdown (RSS, arenas)
• GET /error-types — Error hierarchy for client introspection

Configuration

Environment Variables

Session Persistence

Per-MCP-session niquests.AsyncSession is reused automatically (cookies, connection pool). Sessions are scoped to the MCP session (FastMCP Context), not global; idle sessions are evicted after WEB_SESSION_LIFETIME seconds.

HTTP Mode

export WEB_TRANSPORT=streamable-http
export WEB_HTTP_HOST=127.0.0.1
export WEB_HTTP_PORT=8000
www-search-mcp

Operational endpoints:

• GET /healthz — { "status": "ok" }
• GET /readyz — readiness probe (200/503)
• GET /status — same JSON as web_mcp_status
• GET /error-types — error hierarchy

MCP endpoint:

• POST /mcp — streamable-http

Idle Timeout (stdio)

stdio process auto-terminates after WEB_MCP_IDLE_LIFETIME seconds of inactivity (default 5 min). Set to 0 to disable.

Development

# Setup
git clone https://github.com/naifs/www-search-mcp.git
cd www-search-mcp
uv sync --all-groups

# Format + lint
uv run ruff format src/ tests/
uv run ruff check src/ tests/

# Type check
uv run ty check src/

# Tests
uv run pytest tests/ -q          # parallel (default)
uv run pytest tests/ -q -n0      # sequential

# Security
uv run bandit -r src/

# Build + install
rm -rf dist/
uv build
uv tool install --force dist/*.whl

Troubleshooting

License

MIT