mcp-server-spidy MCP server

a web crawler for claude mcp

Components

Resources

The server implements a simple note storage system with:

• Custom note:// URI scheme for accessing individual notes
• Each note resource has a name, description, and text/plain mimetype

Prompts

The server provides a single prompt:

• summarize-notes: Creates summaries of all stored notes
  • Optional "style" argument to control detail level (brief/detailed)
  • Generates prompt combining all current notes with style preference

Tools

The server implements two tools:

• web-crawl: Crawls a website and saves the results to a text file
  • Takes "url" and "output_file" as required string arguments
  • Fetches the content from the specified URL and writes it to the specified text file

Configuration

[TODO: Add configuration details specific to your implementation]

Quickstart

Install

Claude Desktop

On MacOS: ~/Library/Application\ Support/Claude/claude_desktop_config.json On Windows: %APPDATA%/Claude/claude_desktop_config.json

Usage

To use the web crawling tool, you can call it with the following parameters:

• url: The URL of the website you want to crawl.
• output_file: The name of the file where the crawled content will be saved.

Development

Building and Publishing

To prepare the package for distribution:

1. Sync dependencies and update lockfile:

uv sync

2. Build package distributions:

uv build

This will create source and wheel distributions in the dist/ directory.

3. Publish to PyPI:

uv publish

Note: You'll need to set PyPI credentials via environment variables or command flags:

• Token: --token or UV_PUBLISH_TOKEN
• Or username/password: --username/UV_PUBLISH_USERNAME and --password/UV_PUBLISH_PASSWORD

Debugging

Since MCP servers run over stdio, debugging can be challenging. For the best debugging experience, we strongly recommend using the MCP Inspector.

You can launch the MCP Inspector via npm with this command:

npx @modelcontextprotocol/inspector uv --directory /Users/jrob/repos/spidy run mcp-server-spidy

Upon launching, the Inspector will display a URL that you can access in your browser to begin debugging.