Watch Skill

Give every AI agent eyes for video — and a way to check its own work.

Install · Documentation · Examples · Comparison · Roadmap

Watch Skill turns videos, live streams, meetings, and screen recordings into a searchable, timestamped index. An agent can ask what happened, get an answer that cites the exact moment behind it, and ask again tomorrow without processing the video a second time.

When the video is the agent's own browser or desktop session, THE LOOP closes the circle: record the work, judge it against plain-language criteria, and prove the fix.

uvx --from "watch-skill[standard]" watch-skill setup

_{THE LOOP catching a $NaN total that an end-state screenshot misses, then proving the fix.}

What it does

Available as Claude Code skills, 23 MCP tools, a CLI, a REST API, and native adapters for LangChain/LangGraph, CrewAI, the OpenAI Agents SDK, LlamaIndex, and AutoGen.

Install

One command, any platform, no clone:

uvx --from "watch-skill[standard]" watch-skill setup

That registers the MCP server in every AI agent it finds on the machine, backing up each config first. To wire an agent up by hand:

{ "mcpServers": { "watch-skill": {
    "command": "uvx",
    "args": ["--from", "watch-skill[standard]", "watch-skill", "serve"] } } }

/plugin marketplace add oxbshw/watch-skill
/plugin install watch-skill@watch-skill

pipx install "watch-skill[standard]"

docker run --rm -i -v watch-skill-data:/data ghcr.io/oxbshw/watch-skill serve

curl -fsSL https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.sh | sh

powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.ps1 | iex"

standard is frames, retrieval, and MCP — about 200 MB. watch-skill[all] adds OCR, local Whisper, REST, and the browser THE LOOP drives. watch-skill doctor names anything missing and prints the command that adds it, so starting small is safe.

Coming from claude-video? Your /watch commands and flags work unchanged — see the migration guide.

Then watch a video and ask a follow-up:

watch-skill watch "https://youtu.be/..." "Summarize the important moments."
watch-skill ask <video_id> "When does the demo first fail?"
watch-skill search "pricing decision"       # search every indexed video
watch-skill serve                           # MCP over stdio

Transcription, OCR, and search run locally and need no API key. Visual question answering uses whichever provider you already pay for — Anthropic, OpenAI, Gemini, OpenRouter, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, MiniMax, Moonshot, Z.ai, or Qwen — or nothing at all with a local Ollama model. Anything else that speaks the OpenAI format (vLLM, LM Studio, llama.cpp, LiteLLM, Azure OpenAI, a company gateway) works through the custom provider:

watch-skill setup-vision --provider groq            # or any of the above
watch-skill setup-vision --provider custom \
  --base-url http://127.0.0.1:8000/v1               # your own server

See Getting started for manual installation and Configuration for provider and privacy settings.

Why use it

• Evidence instead of frame dumps. Scene detection and perceptual deduplication spend the frame budget on distinct moments. Answers include timestamps, confidence, and the evidence used to support them.
• Persistent video memory. Analyze once, ask again without downloading or transcribing the same video. Hybrid full-text and vector retrieval works within one video or across the entire library.
• Local-first processing. Original-language captions are preferred, local Whisper is the default fallback, and cloud speech-to-text is opt-in. An Ollama configuration keeps the complete pipeline on the machine.
• Flow verification. THE LOOP records an agent's browser, screen, or window; checks the result against plain-language criteria; and produces before/after proof after a fix.
• Corrections that persist. report_mistake stores a local lesson, applies it to related questions, and turns it into a replayable evaluation.
• Measured cost controls. Text-first answers, semantic caching, configurable token budgets, and explicit cheapest, quality_first, and offline_only policies keep the trade-offs visible.
• Multilingual retrieval. Script-aware OCR routing, Arabic normalization, CJK substring matching, and multilingual embeddings support questions across languages.

The repository includes reproducible cost and perception benchmarks. Product claims in this README link to the relevant implementation notes or testable example rather than relying on unqualified marketing numbers.

Works with your agent

The setup command detects supported clients and updates their configuration with a backup. Manual guides are available for every entry below.

Claude Code	Claude Desktop	Cursor	Codex CLI
Cline	Windsurf	Gemini CLI	VS Code
GitHub Copilot CLI	Kimi Code	Qwen Code	OpenCode
Goose	OpenHands	Kilo Code	Qodo
Agent Zero	OpenClaw	Pi	Hermes

Native tools are also available for LangChain/LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex, and AutoGen; any other framework can use REST or MCP.

Skills, in any agent

The ten skills are the agent-facing layer: they decide when to reach for video, so an agent uses Watch Skill without being told to. They live in a top-level skills/ directory, which is what the open skills ecosystem reads — so they install into any of its 27+ supported agents with one command, not just Claude Code:

npx skills add oxbshw/watch-skill -g

Skills and MCP complement each other: skills carry the judgement about when video is worth watching, MCP carries the 23 tools. Installing both is the full setup, and watch-skill setup does it.

The full compatibility matrix separates machine-tested, machine-configured, and documentation-verified integrations. If your agent is missing, the adapter template provides a short contribution path.

Common workflows

Build a searchable video library

watch-skill batch ./recordings --limit 50
watch-skill library overview
watch-skill library ask "What did the team decide about authentication?"

library ask synthesizes evidence across videos and retains per-video timestamp provenance. The library example demonstrates a question whose answer is distributed across four clips.

Verify an agent's browser work

watch-skill loop start \
  --source "browser:http://127.0.0.1:3000" \
  --criteria "Checkout completes and the total is always a valid currency amount"

The loop captures the full interaction, critiques failures, and records proof after the agent applies a fix — the run shown at the top of this page. Example 14 walks through that transient $NaN bug.

Export an offline report

watch-skill viewer <video_id> --out video-report.html

The generated page contains its frames, transcript, OCR, cached answers, and cited evidence. It has no external runtime dependencies and can be opened without a server.

Examples

The examples progress from a first watch to agent integration, cross-video memory, and self-verification.

See the example catalog for prerequisites, expected output, and a recommended path through all 16 examples.

Architecture

All interfaces call the same Python core. Skills and agent adapters decide when to use Watch Skill; acquisition, perception, transcription, indexing, answering, and verification remain in src/watch_skill.

flowchart LR
    A["Agents and frameworks"] --> S["Skills · MCP · CLI · REST"]
    S --> AC["Acquire"]
    AC --> P["Scenes · OCR · transcript"]
    P --> I[("Persistent index")]
    I --> Q["Answers · extraction · library"]
    I --> L["Lessons and evaluations"]
    V["Browser · screen · stream capture"] --> C["Loop critic"]
    C --> I

Read Architecture for the data model, provider boundaries, and extension points.

Documentation

Development

git clone https://github.com/oxbshw/watch-skill
cd watch-skill
uv sync --extra all
uv run pytest
uv run ruff check .

See CONTRIBUTING.md for test tiers, documentation standards, and the agent-adapter checklist. Security and privacy reports are covered by SECURITY.md.

Listed on

Independent directories that index Watch Skill. They are maintained by their operators, so the details there can lag a release.

• Agent Skills Hub
• Neuralbox