👁️ mcp-local-vision (Local Vision MCP Tool)

Give your local agent eyes. Close the autonomous loop.

mcp-local-vision is the missing link for autonomous local AI. It allows standard MCP agents (Your local agent running on LMStudio/Ollama/vLLM, Claude Desktop, Goose, etc.) to "see" images, screenshots, and visual assets by routing them back through an call to the same inference engine and same model (in case the initial model was a VLM model of course so that it can see and reason about the image thus enhancing your workflows. Most current MCP servers like Playwright, File Access and Screenshot tools are missing the ability to allow the Agentic AI mode to actually see and reason about images and just returning responses that the end user at the GUI level can see but the Agentic Model is blind to, thus hindering the agentic flowIt also supports cloud providers like Anthropic, Gemini, OpenAI, Openrouter etc...

🚫 The Problem: The "Blind Agent" Paradox

You have a cutting-edge local agent. You ask it to "Debug my UI."

1. Agent launches the app.
2. Agent takes a screenshot to check for errors.
3. Agent stops. 🛑

Why? Because standard MCP tools only save screenshots to disk. The agent knows the file exists, but it cannot see the pixels. It is blind to its own actions.

✅ The Solution: The Visual Loop

mcp-local-vision bridges the file system and your inference engine.

1. Agent takes a screenshot via Playwright/Puppeteer.
2. Agent calls analyze_image(path).
3. mcp-local-vision instantly pipes the raw pixels to your already hosted model (no extra host, VRAM/RAM usag, just efficient routing/loop in case the initial model supports Vision, or choose any other endpoint where a VLM is hosted and it works too.
4. Agent receives a detailed description (from it-self sometimes! for a full descroption on the image which can allow it to proceed further ("I see a 404 error on the dashboard").
5. Agent takes the next action to fix it. The loop continues.

🚀 Features

• 100% Local Privacy: Your screenshots never leave your machine if you want to! (when using `local hosted models such as LMStudio, llama.cpp, vllm, Ollama or others.)
• Multi-Provider Support: Plug in Anthropic (Claude), Google Gemini, OpenRouter, or OpenAI if you need cloud power.
• Smart Fallbacks: Automatically handles "Reasoning" models (like DeepSeek-R1) and connection errors.
• Robust Validation: Checks for valid image files before analysis.
• Zero-Config Install: Run instantly with uvx.

⚡ Quick Start

1. Minimal Setup (Local)

No cloning required. If you use uv, you can run this instantly:

uvx mcp-local-vision

Alpha quick start for LM Studio (local loop)

1. Start LM Studio local inference server on http://localhost:1234/v1 with a vision model (e.g., llava, llama-3.2-vision, or whichever you loaded).
2. Set LOCAL_MODEL (optional; defaults to llava), plus any LOCAL_INFERENCE_URL/LOCAL_INFERENCE_KEY overrides.
3. Run from source: python src/mcp_local_vision/server.py (ensure PYTHONPATH=src) or use the installed entrypoint: uvx mcp-local-vision.
4. Point your MCP client to the command above; call analyze_local_image("/abs/path/to/image.png").

2. Configuration (Claude Desktop / Agent Config)

Add this to your claude_desktop_config.json or mcp_server_config.json.

Method 1: Using uvx (Recommended)

This is the Python equivalent of npx. It automatically downloads and runs the latest version of the package in an isolated environment.

{
  "mcpServers": {
    "local-vision": {
      "command": "uvx",
      "args": ["mcp-local-vision"],
      "env": {
        "LOCAL_INFERENCE_URL": "http://localhost:1234/v1"
      }
    }
  }
}

Note: You can also use the explicit uv command if uvx is not in your PATH:

    "local-vision": {
      "command": "uv",
      "args": ["tool", "run", "mcp-local-vision"],
      ...
    }

Method 2: Manual / Development (Using local venv)

If you have cloned the repo and want to run it from source (like the example you provided):

{
  "mcpServers": {
    "local-vision-dev": {
      "command": "/absolute/path/to/your/custom/venv/bin/python",
      "args": [
        "/absolute/path/to/mcp-local-vision/src/mcp_local_vision/server.py"
      ],
      "env": {
        "PYTHONPATH": "/absolute/path/to/mcp-local-vision/src",
        "LOCAL_INFERENCE_URL": "http://localhost:1234/v1"
      }
    }
  }
}

Note: Replace /absolute/path/to/... with your actual paths.

3. Backend Configuration

Option A: Local (LM Studio / vLLM) - Default

Set LOCAL_INFERENCE_URL in the env section (see above).

Option B: Ollama

      "env": {
        "OLLAMA_URL": "http://localhost:11434/v1",
        "OLLAMA_MODEL": "llava"
      }

*And pass `provider='ollama'` when calling the tool.*

*Tip: If your agent is already using a Vision Logic Model (VLM) like `llama3.2-vision` or `qwen2.5-vl`, set `OLLAMA_MODEL` to that same model name to create an efficient self-loop without loading a second model.*

#### Option C: Cloud Providers (Anthropic / Google / OpenRouter)
```json
      "env": {
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GOOGLE_API_KEY": "AIza...",
        "OPENROUTER_API_KEY": "sk-or-..."
      }

🛠️ Usage

The server exposes one tool: analyze_local_image.

Arguments:

• file_path (string): Absolute path to the image file.
• question (string): What to ask the model (Default: "Describe this image in detail").
• timeout (int): Timeout in seconds (Default: 30).
• provider (string): One of local (default), ollama, openai, anthropic, google, openrouter.

Example User Query: "Look at the screenshot at /tmp/screen.png using the anthropic provider and tell me if the tests passed."

❓ Troubleshooting

The "Self-Call" Architecture

If you are using an agent (like a VLM in LM Studio) to use this tool, remember:

1. The Agent sends a request to MCP.
2. MCP reads the file.
3. MCP sends the image BACK to LM Studio (or Ollama) for analysis.
4. LM Studio returns the description.
5. MCP returns the description to the Agent.

Critical: If you are using the LM Studio GUI to chat, you must ensure the Local Inference Server is also turned ON and running on port 1234 (or your configured port). The GUI itself does not expose the API unless the server is started.

Common Errors

• "Connection Refused": Check that your local server (LM Studio/Ollama) is actually running.
• "Model returned empty response": You might be using a text-only model (like Llama 3) instead of a Vision model (like Qwen2-VL or LLaVA).