perception-mcp 0.1.0

Perception & verification MCP server for AI agents: media_assert (declarative pass/fail QC for generated media), image diff (pixelmatch + SSIM), readable text tiles for small print & handwriting, video tools (frames on demand, scenes, QC defects, pacing dynamics, attention heatmaps, scene diff), and full audio DSP (pitch, BPM, key, LUFS, spectrograms).

perception-mcp

Perception & verification MCP server for AI agents — the measuring instruments a text-and-image model lacks. Where a model can only look at media, this server lets it measure, compare and verify: local recognized algorithms (no cloud APIs), compact JSON answers, and PNG renderings the model can look at. The successor and superset of audio-dsp-mcp.

The idea

An agent that generates or edits media works blind: it can't hear the podcast it produced, can't tell whether two screenshots differ by a real regression or antialiasing noise, can't read the exact color of a button. Every tool here follows one formula:

opaque file → compact JSON with numbers → optional PNG for the model's vision → an honest "not sure" when the signal isn't there.

Tools

media_assert — declarative QC (the headline)

One call turns the whole server into a test harness for media:

media_assert("episode.mp3", expect={
  "duration_s": [1790, 1810],
  "lufs": [-17, -13],
  "no_clipping": true,
  "no_silence_longer_s": 2.0
})

→ {"passed": false, "summary": "3 of 4 checks passed (1 failed)", "checks": [{"check": "lufs", "status": "fail", "actual": -9.2, ...}]}

Checks run only for what you ask, files are decoded once, an unknown check name returns the full supported list (instant self-correction), and a check that doesn't apply to the file type is reported skipped, not failed. Generate → assert → regenerate.

Video files get their own keys (needs ffmpeg): resolution, fps, has_audio, no_black_longer_s, no_freeze_longer_s, no_silence_longer_s, min_sharpness, duration_s.

image ( [image] extra )

• compare_images — perceptual pixel diff (pixelmatch YIQ metric)
  • SSIM structural score, with a global-shift compensation via phase correlation (a 1-px scroll no longer lights up the whole diff), antialiasing separation, and changed-pixel clusters with exact bounding boxes. Returns an A | B | highlight PNG panel. Tolerance modes: strict / normal / layout.
• measure_image — exact pixel colors (hex/rgb/CSS name) at given points, pixel distances between points, dominant palette (k-means in CIELab).
• zoom_grid — a microscope: magnify a region with a labeled coordinate grid in absolute source coordinates, so tiny text, 1-px borders and exact positions become visible and referenceable.

ocr ( [ocr] extra )

• extract_text_boxes — printed text with exact pixel coordinates per line (RapidOCR: PaddleOCR's DBNet + CRNN via ONNX, models ship in the wheel — no downloads, no cloud).
• prepare_text_tiles — the small-print / handwriting reader. A vision LLM fails on tiny text because tokenization gives each glyph too few pixels; this tool fixes that with classic document preprocessing — CLAHE contrast, projection-profile deskew, Sauvola ink mask, line segmentation — and returns each line as a big magnified labeled strip for the calling model to read itself. No recognition model in the loop, so it works for any script and any handwriting the calling model can read.
• check_contrast — WCAG 2.x audit of a screenshot: per text line, exact W3C contrast ratio + AA/AAA pass/fail.
• media_assert keys: contains_text, min_contrast_ratio.

audio ( [audio] extra — the full audio-dsp-mcp toolset )

detect_pitch, estimate_tempo, detect_key (+ Camelot), analyze_loudness (LUFS, ITU-R BS.1770), detect_silence, analyze_audio, compare_audio, render_spectrogram, render_waveform, describe_audio, get_metadata; with the [youtube] extra get_youtube_transcript, with [transcribe] local Whisper transcribe_audio.

video ( no extra — just ffmpeg on PATH )

• describe_video — the whole video in one call: stream facts, shot structure (cuts, avg shot length), audio profile, speech transcript (with [transcribe]), and a contact sheet with a timestamped frame per scene.
• get_frames — SEE the footage: exact timestamps, an even sweep (every), or a storyboard of one moment (around=[t, span, fps]); many frames pack into one timecoded contact sheet.
• check_video — QC with exact timestamps: black stretches, frozen picture, silent audio (ffmpeg blackdetect / freezedetect / silencedetect), soft focus (Laplacian variance).
• video_dynamics — pacing on one timeline: cut rhythm, motion energy, EBU R128 momentary loudness, hook metrics for the opening seconds (first cut, cold-start flags) + a chart image.
• video_heatmap — WHERE things happen: motion accumulation and spectral-residual saliency (Hou & Zhang) drawn over a real frame, with a 3x3 grid breakdown for crop/caption decisions.
• compare_videos — scene-level diff of two cuts (Needleman-Wunsch alignment + SSIM): identical / changed / added / removed scenes.
• detect_scenes — shot boundaries as timestamps.

Every analysis tool takes offset_s / duration_s, so any part of a long file is reachable; reported timestamps are absolute.

Sources: every tool takes a local path, a direct http(s) URL (25 MB cap), or a base64 data URI; with [youtube], platform page URLs resolve via yt-dlp. Note for calling models (also served as MCP instructions): a chat attachment you merely see is not a file you can forward — pass the saved path when the client provides one (Claude Code does), otherwise ask the user for a path or a public URL. All failures come back as an "error" field — the server never crashes the conversation.

docs ( [docs] extra )

• compare_pdf — page-level visual diff of two PDFs (pypdfium2 render → the same perceptual diff as screenshots): layout shifts, swapped images, font substitution; diff panels for changed pages.
• check_pdf — will it open everywhere: structural validation (pikepdf/qpdf), encryption, non-embedded fonts, pages that crash rendering, blank pages.

fonts ( [fonts] extra )

• render_font_specimen — SEE a typeface and verify it: rendered specimen sheet (pangrams for every script the font covers) + exact cmap coverage ("cyrillic": "complete" / "partial (74%)" / "none"), metrics, variable axes.

mesh ( [mesh] extra )

• render_model — seven orthographic views of a 3D model on one sheet (GLB/GLTF/STL/OBJ/PLY), pure-CPU rendering, works headless.
• check_mesh — printability QC: watertightness, winding, degenerate faces, volume, extents, printable verdict with reasons.
• compare_mesh — geometric diff: ICP alignment + two-sided point-to-surface distances (chamfer / Hausdorff) + overlay render.

Install

Core is lean; add what you need: [image], [ocr], [audio], [youtube], [transcribe], [docs], [fonts], [mesh], or [all].

Claude Code

claude mcp add perception -- uvx --from "perception-mcp[all]" perception-mcp

Claude Desktop / any MCP client

{
  "mcpServers": {
    "perception": {
      "command": "uvx",
      "args": ["--from", "perception-mcp[all]", "perception-mcp"]
    }
  }
}

Until the PyPI release, use --from "git+https://github.com/egorthinks/perception-mcp".

Roadmap

MCP sampling mode, A/V sync detection, mesh wall-thickness analysis. See PLAN.md in the repository.

License

MIT