microscopy-llm-profiler

Deterministic microscopy image analysis for AI agents, over MCP.

Point your Claude / Cursor / Codex agent at a microscopy image and it does the analysis for you — segment, count, measure, localize, colocalize — by calling 51 deterministic MCP tools. Every number comes from a named tool (segmenter + metric), traceable and reproducible — never an LLM guess. No browser, no separate analysis API key: your agent is the orchestrator; Python does the math.

Think of it as Fiji/ImageJ, driven in plain language — "profile this ND2", "count the rods on the DIC channel", "give me per-cell bipolar vs septal localization" — with the rigor of a scripted pipeline.

Why

• Trustworthy by construction. The engine is deterministic; the agent chooses which tool and interprets overlays, but it cannot invent a count. Every result is a segmenter + metric you can re-run.
• QC-first workflow. Metadata, saturation, SNR, and focus checks run before any measurement, so you catch bad data instead of counting noise.
• Formats scientists actually use. Native .nd2 (multi-channel, by channel name), .tif/.tiff, .png.

What's inside

• Segment & count — nuclei, puncta/vesicles, condensates, and rod bacteria (DIC-guided), with an evaluate-and-retune quality loop for counts you can trust.
• Segmenters — opencv_watershed (blobs, deterministic, no GPU), dic_rod (rods on DIC/brightfield), plus optional deep-learning cellpose / stardist_2d.
• Measure — per-object tables (area, intensity, shape), colocalization (Pearson/Manders), spatial cross-correlation, subcellular localization (membrane / polar / bipolar / septal / diffuse).
• Agent-in-the-loop — threshold_step / segment_step / outline_step for one-iteration-at-a-time visual review on hard images.
• A /micro harness prompt that boots the analysis workflow, shipped inside the package (no files to copy).

Install (no repo clone)

# A) run on demand with uv — nothing installed globally
claude mcp add microscopy-profiler --scope user -- uvx microscopy-llm-profiler

# B) or pip install, then register + install the /micro skill
pip install microscopy-llm-profiler
microscopy-profiler-setup            # writes the /micro + /welcome skills, prints the mcp-add line
claude mcp add microscopy-profiler --scope user -- microscopy-profiler

Restart Claude Code, run /mcp to confirm microscopy-profiler is connected, then type /micro to boot the harness (or use the built-in MCP prompt /mcp__microscopy-profiler__micro — it ships in the package, no local files needed).

Other clients: the server speaks stdio MCP, so it works in Cursor, Claude Desktop, and Codex too — point their MCP config at the microscopy-profiler command. microscopy-profiler-setup --print-config prints the JSON snippet.

Optional extras (deep-learning segmenters / Fiji macros / VLM judge):

pip install "microscopy-llm-profiler[cellpose]"   # or [stardist], [fiji], [vision]

Use

Just describe the task with an absolute image path:

Profile /abs/path/to/sample.nd2 — channels, pixel size, and any quality issues (saturation, low SNR, out of focus) before we measure anything.

In /abs/path/to/sample.nd2, use the DIC channel for cell boundaries and the GFP channel for signal. Count rods and give per-cell bipolar / septal / diffuse localization with a histogram.

Requirements: Python ≥ 3.11. Deep-learning segmenters benefit from a GPU but aren't required — the core watershed/rod path is CPU-only and deterministic.

Data handling & security

Designed to be safe on unpublished / sensitive research data. Defaults are private:

• Nothing leaves your machine by default. The engine is fully local. The only off-box path is the optional VLM quality-check, which is doubly gated — it sends image pixels to the Anthropic API only if you set both ANTHROPIC_API_KEY and MICROSCOPY_PROFILER_ALLOW_VLM=1. Data sent there is subject to Anthropic's terms. Leave it unset to stay 100% local.
• Telemetry is OFF by default. Product/usage telemetry is written only if you opt in with MICROSCOPY_PROFILER_TELEMETRY=1; then it goes to local JSONL under ~/microscopy_profiler_runs/telemetry/ with file paths/emails redacted best-effort. It stays local unless you also set a collector URL (opt-in "phone home", MICROSCOPY_PROFILER_TELEMETRY_UPLOAD_URL) — a deliberate second switch. Even then, only redacted events are sent, and free-text queries are stripped unless you set MICROSCOPY_PROFILER_TELEMETRY_UPLOAD_TEXT=1. See SECURITY.md for all flags.
• Run provenance (manifests, overlays) is written under ~/microscopy_profiler_runs/ and includes the input file path. It has no auto-expiry — relocate it with MICROSCOPY_PROFILER_RUNS=/path (and telemetry with MICROSCOPY_PROFILER_TELEMETRY_DIR) to a scratch/encrypted location and purge periodically for sensitive data.
• run_fiji_macro is arbitrary code execution (ImageJ macros can spawn shells) and is disabled unless you set MICROSCOPY_PROFILER_ALLOW_FIJI_MACROS=1 in a trusted environment. Every other tool is deterministic and non-privileged.

Locked-down profile for clinical/sensitive data: don't set ANTHROPIC_API_KEY / MICROSCOPY_PROFILER_ALLOW_VLM, leave telemetry off, and point the runs dir at a scratch location you control.

License

PolyForm Noncommercial 1.0.0 — free to use, modify, and share for any noncommercial purpose (research, education, nonprofits, personal/hobby use). Commercial use is not permitted — you may not use this software to build a commercial product or business. See LICENSE. For a commercial license, contact the maintainers.