Run against three servers from the official modelcontextprotocol/servers repo:

• src/fetch — 100% / A. Clean.
• src/git, src/time — flagged as parse errors, not false passes. Both use Python match statements (3.10+ syntax); mcp-doctor's AST parser follows the grammar of whatever Python interpreter runs it, so under Python 3.9 those files can't be parsed. Rather than silently skip them and report a misleadingly clean score, mcp-doctor surfaces this as an explicit error: "N file(s) could not be parsed and were skipped." Run it under Python ≥3.10 to analyze those files correctly.

Later spot-checked against 4 more real, in-the-wild servers (awslabs' aws-documentation-mcp-server, mcp-google-ads, sv-excel-agent, and Home Assistant's ha-mcp, an 88-tool server). That run caught two real precision bugs: the secret scanner was flagging test fixtures and identifier-style constant names (SERVICE_GET_CALLER_TOKEN = "get_caller_token") as hardcoded credentials, and the param-docs check didn't recognize Annotated[T, Field(description=...)] — a completely valid, schema-level way to document a parameter — as documentation at all, since it only looked for a docstring Args: section. Both fixed.

A maintainer on ha-mcp reviewed the resulting report in detail and pushed back further, correctly: the param-docs check still missed descriptions reached through a shared, cross-file type alias (Annotated[..., Field(description=...)] assigned to a name and imported elsewhere) and prose under non-Args: headings (e.g. **Parameters:**, including bulleted - param: ... lines), and — more importantly — the error-handling check's own message was wrong. It claimed a missing try/except lets a raw traceback leak through the MCP transport; FastMCP's call_tool dispatcher actually wraps every call and converts any exception into a structured error regardless, which the pushback prompted me to verify directly against FastMCP's source. Both the alias/heading gaps and the error-handling message are now fixed — see homeassistant-ai/ha-mcp#2324 for the full exchange.

The maintainer offered to leave a follow-up issue open if it were grounded in the actual spec and FastMCP's own guidelines rather than another pass of the same heuristics. Read the current spec's Tools page end to end looking for exactly that: one concrete, checkable gap emerged — the normative Tool Names section (length, character set, uniqueness), which mcp-doctor didn't check at all — now added. Checked it against ha-mcp's real 88 tool names before claiming anything: all of them already comply, so this doesn't reopen anything there — it's a real gap closed for the next server that isn't as careful, not a finding to hand back.

A third real-world pass against mendableai/firecrawl-mcp-server (7k+ stars, TypeScript) found a genuine gap: it reported 0 tools on a 28-tool server. The repo registers every tool through the community fastmcp package's server.addTool({ name, description, parameters, execute }) — a single-object call shape mcp-doctor's TS analyzer didn't recognize at all, having only ever seen the official SDK's positional-arg registerTool/tool styles. Added support for it (verified against fastmcp's own docs, not just this one repo's usage), re-ran, and got a real 84%/B report on all 28 tools. Also surfaced a duplicate-name warning (firecrawl_search registered twice) that turned out to be a false positive: the two registrations are read from source, but the code documents and structurally enforces that they're mutually exclusive by runtime profile and land on separate server instances — the check has no way to know that statically. Left as-is rather than filing anything upstream; see Known limitations below.

A fourth pass against exa-labs/exa-mcp-server (TypeScript) found the TS analyzer silently dropping tools named with the common toolName || "default-name" optional-override idiom — treated as fully dynamic (like a genuinely unattributable name from a loop) rather than resolved to the literal fallback actually used at runtime. Its two headline tools, web_search_exa and web_fetch_exa, were invisible: 9 tools reported instead of the real 11, still showing a false 100%/A. Fixed by unwrapping a || binary expression to its literal right-hand side during name resolution, with a regression test confirming a genuinely dynamic name (t.name from a loop) still correctly falls through to skipped.

A fifth pass against stickerdaniel/linkedin-mcp-server (Python) found 13 of 19 tools falsely flagged as undocumented. All 13 use FastMCP's @mcp.tool(exclude_args=[...]) to keep an internal-only parameter out of the exposed schema (verified against FastMCP's own docs: an excluded arg literally can't be passed by an agent) — every one had complete Args: docs for every parameter an agent can actually pass, but the param-docs check was still requiring documentation for the excluded one too. Fixed by excluding exclude_args names from the parameter count and doc requirement entirely; 93%/A on that repo corrected to the true 100%/A.

That same session, a sixth pass against atilaahmettaner/tradingview-mcp (39 tools) turned up something worth fixing in the tool itself rather than a new false positive: the error-handling check's known delegation blind spot (below) had by then shown up on two separate real repos (firecrawl and this one — market_sentiment delegates through analyze_sentiment → _get_articles → _request, three calls deep across files, before reaching the actual try/except around the network call). Built a repo-wide, name-based resolution (same simplification already used for the Field alias registry — resolved by function name, not by which file it's imported from) that transitively follows a tool's direct calls to locally-defined functions, so a tool that delegates to a helper that itself has real error handling — however many calls deep — is no longer flagged. Re-checking the same repo surfaced one more real gap in the fix itself: compare_strategies delegates through _compare_strategies, a from strategies import compare_strategies as _compare_strategies alias — the registry was keyed by the def name (compare_strategies), so the aliased call site didn't resolve. Added alias resolution (from x import y as z mapped back to y) so the registry itself understands the alias. Verified it doesn't just widen the check: a tool calling only unhandled or genuinely external code is still correctly flagged, and one real remaining gap was left honestly undone rather than papered over — financial_news passes its helper as an argument to asyncio.to_thread(fetch_news_summary, ...) rather than calling it by name directly, a different idiom this resolution doesn't cover; still flagged.

A seventh pass against GLips/Figma-Context-MCP (15k+ stars, TypeScript) found the TS analyzer reporting 0 tools on a repo with two well-built, widely-used tools — a false 80%/B with no tools listed at all. Root cause: both are registered as server.registerTool(getFigmaDataTool.name, { description: getFigmaDataTool.description, ... }, handler), where getFigmaDataTool is an exported { ... } as const object literal defined in a separate file — a cross-file member-expression property lookup, not a same-file const reference, which is all the resolver previously understood (documented below as a known limitation until now). Fixed by adding a repo-wide, name-based registry of const NAME = {...} object literals (same simplification as the Python side's Field-alias registry) and teaching the resolver to follow member-expression property access into it, plus unwrap TypeScript's as const/satisfies assertions along the way. Re-verified: 0→2 tools found, one clean pass and one correctly-flagged real gap — download_figma_images's description is built by a runtime function call (getDescription(imageDir), not a property access), which is genuinely dynamic and correctly still reported as unresolvable rather than guessed at. 2 new regression tests (49 total), confirmed no regression on the exa-mcp-server and firecrawl-mcp-server repos from earlier passes.

An eighth pass against hangwin/mcp-chrome (12k+ stars, TypeScript) found the TS analyzer reporting 0 tools on a repo with 27 real, well-documented ones — a hollow false 100%/A. Root cause was architectural, not a small parsing gap: the repo builds its server on the low-level Server SDK, wiring up server.setRequestHandler(ListToolsRequestSchema, () => ({ tools: [...TOOL_SCHEMAS, ...dynamicTools] })) with a static array of raw-JSON-Schema Tool objects, rather than any of the registerTool/.tool()/.addTool() call-site styles already supported. Unlike the earlier playwright-mcp/XcodeBuildMCP cases (correctly ruled out as out of scope — their tool definitions live outside the repo entirely), this repo's tool metadata is fully present and staticaly analyzable, so it was worth adding real support rather than declining: a new code path finds the handler's { tools: [...] } response literal, follows ...constArraySpreads (repo-wide, via the same registry used for member-expression resolution) into their elements, and checks each tool's description and raw-JSON-Schema properties[x].description — while not checking error handling for this style, since there's no per-tool handler closure to inspect (one generic dispatcher serves every tool by name here, proxying over native messaging to the Chrome extension process where the real logic lives). Also found and fixed two bugs in the new code while verifying it against the real repo: the same static array is spread into two separate transport entrypoints (stdio and HTTP), which without dedup reported each tool twice; and tool locations were initially reported at the wrong file (the call site instead of the array's actual definition site). 2 new regression tests (51 total). Verified: 0→27 tools found, a real 100%/A this time.

A ninth pass against BeehiveInnovations/pal-mcp-server (11k+ stars, Python) surfaced a real bug in mcp-doctor's own low-level-Tool-constructor check, not just another architectural gap. The repo defines each of its 17 real tools as a class (ChatTool, DebugIssueTool, etc.) registered in a TOOLS dict, and builds the actual MCP Tool(...) objects in a loop at list-time — Tool(name=tool.name, description=tool.description, inputSchema=tool.get_input_schema()) — so the one Tool(...) call site mcp-doctor found had a genuinely dynamic name it couldn't resolve. Rather than skip it (the correct, established behavior for a dynamic name — see the TS analyzer's identical handling of a t.name loop variable), the Python-side check fell back to a fabricated "<unnamed>" tool with a nonsensical "no description" error, worse than reporting nothing. Fixed by skipping instead of guessing, with a regression test (52 total). Full support for this class-based tool pattern — resolving get_name()/get_description() across a real Python class hierarchy, and introspecting get_input_schema() when it's built by imperative code rather than a literal dict — was correctly left undone rather than forced: unlike the TS static-array case, this would mean walking method resolution across base classes and interpreting arbitrary schema-building code, a much larger and more failure-prone undertaking than a scoped fix; see Known limitations below.

A tenth pass against wonderwhy-er/DesktopCommanderMCP (9k+ stars, TypeScript) — also on the low-level Server SDK's setRequestHandler(ListToolsRequestSchema, ...) style added in the eighth pass — found two more real gaps in that new support, plus surfaced one genuine, real gap in the target repo itself. First: the handler returns { tools: filteredTools }, where filteredTools = allTools.filter(tool => shouldIncludeTool(tool.name)) — a runtime filter over the real base array. The resolver didn't know how to look through a .filter(...) call, so the whole 26-tool list was invisible (0 tools, a hollow 80%/B). Fixed by resolving straight through .filter(...) to its base array — filtering never invents or changes a tool's definition, only its runtime visibility, so for audit purposes the base array is the right thing to check. Second: every tool's description is a template literal with one interpolated suffix (`Get the complete server configuration... ${CMD_PREFIX_DESCRIPTION}`) — _string_value was discarding the entire string whenever a template literal had any ${...}, which, once tools were visible at all, turned into 26 false "no description" errors on a repo that documents its tools extensively. Fixed to join the literal fragments and drop only the interpolated part, so real (if partial) description text is no longer thrown away just because part of it is dynamic. Re-verified: 0→26 tools, real 98%/A. Along the way, also added support for the zodToJsonSchema(SomeArgsSchema) idiom (the well-known zod-to-json-schema package) — inputSchema here isn't a raw JSON-Schema literal but a runtime conversion of a real Zod schema, so param docs unwrap to that Zod schema rather than going blind. That unwrap surfaced a genuine, real gap in the target repo, not a mcp-doctor false positive: none of its Zod schemas use .describe(...) on any parameter (their docs live entirely in prose on each tool's top-level description instead) — correctly left as an accurate finding rather than filed upstream, since documenting per-tool instead of per-param is a defensible stylistic choice, not a clear bug. 2 new regression tests (54 total); no change on the four previously-verified TS repos (mcp-chrome, exa-mcp-server, firecrawl-mcp-server, Figma-Context-MCP), re-checked live.

An eleventh pass against MODSetter/SurfSense (16k+ stars, Python; audited via its surfsense_mcp subdirectory, the actual MCP server component of a larger full-stack app) found every one of its 28 real tools false-flagged for missing error handling — a suspicious 100% hit rate on an actively-maintained repo, worth investigating rather than trusting. The cause: this codebase's entire error-handling architecture is built on delegating to object methods — a tool calls a bare helper function, which calls client.request(...) or context.resolve(...), where the real try/except actually lives — but the delegation registry's _direct_call_names only ever recognized bare helper(...) calls (ast.Name), never obj.method(...) (ast.Attribute), so none of that chain was ever followed, even though the registry already indexes methods by name (ast.walk doesn't distinguish a class body from module level — only the call-site extraction was the gap). Fixed by also collecting attribute-call names, resolved through the same name-based registry already used for bare functions — a consistent extension of an already-accepted simplification, not a new category of imprecision, though a generic method name (get, run, close) now carries a higher name-collision risk than a distinctively-named bare function, called out explicitly in Known limitations below. Re-verified: 89%→98%/A, all 28 tools correctly cleared. 1 new regression test (55 total); no regression on ha-mcp (still 88 tools/97%/A), pal-mcp-server (still 0 tools/100%/A — unaffected, correctly), or tradingview-mcp (still 39 tools/99%/A, the same two genuine remaining gaps — top_losers's missing param docs and financial_news's helper-passed-as-argument idiom — still correctly flagged, not incorrectly cleared).

A twelfth pass covered three more real repos. 0x4m4/hexstrike-ai (11k+ stars, Python, 151 tools) came back clean on the mcp-doctor side — a useful data point on its own, since it's the largest real repo audited so far and every finding held up on manual verification: a genuine duplicate tool name (two different @mcp.tool()-decorated functions both named httpx_probe, so FastMCP silently lets the second shadow the first — the tech-detection variant is unreachable) and a genuine bare except: swallowing a JSON-parse error. Correctly held off filing either upstream: the maintainer hasn't merged any of the last 10 PRs and hasn't committed in a month, so the odds of engagement are low enough that it wouldn't produce the kind of real back-and-forth that made the ha-mcp arc valuable. idosal/git-mcp (8k+ stars, TypeScript) reports 0 tools correctly, not a bug: it's a multi-tenant service that generates a different tool set per proxied GitHub repo (different handler classes — DefaultRepoHandler, ThreejsRepoHandler, etc.) — there's no one fixed catalog to audit, the same category of correct decline as playwright-mcp/XcodeBuildMCP, just for a different underlying reason.

The pass against CursorTouch/Windows-MCP (6k+ stars, Python) did find a real bug, and a suspicious one: every one of its 19 tools was false-flagged for undocumented parameters — another 100% hit rate worth investigating rather than trusting (the same instinct that paid off on the SurfSense pass). The cause: every tool takes a ctx: Context = None parameter (FastMCP's context-injection convention), and the parameter counter didn't know that FastMCP strips Context-typed parameters from the tool's exposed schema entirely before it's ever built — verified directly against fastmcp's own source (function_parsing.py's without_injected_parameters). So a tool with every real parameter fully documented via Annotated[T, Field(description=...)] still failed the count, since the always-undocumented, always-uncountable ctx param was being counted as a real one needing docs. Fixed by giving Context-typed parameters (including Context | None/Optional[Context]) the same treatment self/cls/exclude_args already get. Re-verified: 89%→90%/A on Windows-MCP, with the tools that really were fully documented now correctly cleared and the ones that genuinely aren't (documented only in prose on the tool description, not per-parameter) still correctly flagged. 1 new regression test (56 total); also fixed a real, previously-invisible improvement on ha-mcp (97%→98%, same root cause, unnoticed until now) with no regression on pal-mcp-server, tradingview-mcp, or SurfSense.

A fourteenth pass against ChromeDevTools/chrome-devtools-mcp (50k★, TypeScript, the official Google Chrome DevTools MCP server) found the largest single gap yet: 0 of 61 real tools detected, a hollow false 100%/A on the highest-star repo audited so far. Root cause: every tool here is built through a defineTool(...)/definePageTool(...) wrapper factory — either called directly with an object literal, or with an arrow function that returns one (defineTool(args => { return {...}; })) — a registration shape none of the previously-supported call styles (registerTool/.tool()/.addTool()/ListToolsRequestSchema) matched at all. Added direct support: the analyzer now extracts the tool-definition object either from the literal argument or from the top-level return of a factory function, then reuses the existing description/schema/handler checks unchanged. While verifying the fix against the real repo, two more real, independent bugs surfaced and were fixed in the same pass: install_pwa and three sibling tools build their descriptions via JS string concatenation ("..." + "...") across multiple lines, which the analyzer's string-literal resolver didn't handle, producing a false "no description" error on a tool that was in fact well-documented; and several tools reference a shared, top-level const Zod schema (e.g. manifestId: manifestIdSchema) rather than writing .describe(...) inline, which the param-docs check didn't resolve through, producing false "undocumented parameter" warnings on parameters that were correctly documented in their shared definition. Fixed both (string concatenation joins recursively; a bare identifier used as a schema property is now resolved to its const definition, the same simplification already used elsewhere, before checking for .describe(...)). Re-verified: 0%→61 tools found, 88%→93%/A after both follow-on fixes. 7 new regression tests (63 total); re-checked firecrawl-mcp-server and Figma-Context-MCP live to confirm no regression on the identifier-resolution and string-handling code paths this touched.

A fifteenth pass against czlonkowski/n8n-mcp (23k★, TypeScript) reported 0 tools, and this one was worth digging into rather than trusting: the repo's ListToolsRequestSchema handler does return { tools }; — JS shorthand property syntax — which the analyzer's object-property reader had never handled at all (it only recognized { tools: tools }'s explicit key: value form), so the whole handler was silently skipped. Fixed that gap (a shorthand_property_identifier node is now treated the same as an identifier for resolution purposes) — a small, generically useful fix, since { x } shorthand is an extremely common JS idiom well beyond this one repo. Verifying it against n8n-mcp surfaced a second, more serious bug in the process: the local tools variable this now tried to resolve turned out to collide with a completely unrelated tools local variable declared in a different function 1,200 lines away in the same file — and the existing name-based const registry, being scope-blind, silently resolved to whichever declaration it happened to walk last (this.repository.getAITools(), a genuinely dynamic method call), not the real one. That's a worse class of bug than under-reporting: a wrong answer stated as fact. Fixed by treating any name declared more than once in a single file as ambiguous and leaving it unresolved entirely, the same safe "don't guess" fallback already used for genuinely dynamic values. 2 new regression tests (65 total); re-verified chrome-devtools-mcp, firecrawl-mcp-server, and Figma-Context-MCP all still report their previously-verified numbers with no change. n8n-mcp itself still correctly reports 0 tools after both fixes — not because of a parser gap any more, but because its real registered tool set is genuinely assembled at runtime through several chained, conditional .push()/.map() operations (env-var-gated inclusion, client-detection-based description rewriting) that can't be safely resolved without risking exactly the kind of silent-wrong answer just fixed above; see Known limitations below.

A sixteenth pass added a new language rather than another fix: Go support, built against real source rather than assumed. Two real, popular Go MCP servers were studied before writing any analyzer code — the official modelcontextprotocol/go-sdk's own examples (which document, in a code comment, that the SDK infers a tool's parameter schema from json/jsonschema struct tags on the handler's argument type) and github/github-mcp-server (32k★, whose own in-repo migration guide documents the older mark3labs/mcp-go fluent-builder style it's moving away from — real enough to exist, but not verified closely enough here to support yet; see Known limitations). Built support for the official SDK's generic mcp.AddTool(server, &mcp.Tool{...}, handler) style, covering both ways parameters get documented in the wild: an explicit InputSchema, and the struct-tag-inferred form. First real-world test, xpzouying/xiaohongshu-mcp (15.6k★), reported 0 of 18 tools — not a bug in the new analyzer's core logic, but a gap in handler resolution: every single tool in this repo wraps its handler in a withPanicRecovery("name", func(...) {...}) helper call rather than passing it directly, which the first version didn't unwrap. Fixed by resolving through call-expression wrappers generically (capped at 3 hops) rather than special-casing this one helper name. Re-verified against the real repo, cross-checking several tools' resolved parameter counts directly against their struct definitions by hand rather than trusting a clean-looking report: 0→18 tools, 99%/A. Also surfaced, and left alone rather than silently absorbed: two tools have complete, correctly-formed Chinese-language descriptions (e.g. "检查小红书登录状态", 9 characters, a full sentence) that the existing <10 chars → likely just restates the name heuristic — shared by all three language analyzers, not new to Go — flags as too short, since it was calibrated against English character density. Documented as a real, newly-discovered cross-language limitation rather than fixed blind, since a proper fix needs real thought about how to fairly weigh description length across writing systems. 12 new regression tests (77 total).

A seventeenth pass closed the gap the sixteenth pass had deliberately left open: mark3labs/mcp-go's fluent-builder style, real but unverified at the time. Found a second, real, live Go MCP server using exactly that style — korotovsky/slack-mcp-server (1.8k★) — and cloned mark3labs/mcp-go itself to verify the exact API against its source before writing any detection code, rather than trusting the earlier assumption. Confirmed: s.AddTool(mcp.NewTool(name, mcp.WithDescription(...), mcp.WithString("x", mcp.Required(), mcp.Description(...)), ...), handler), where every parameter is declared inline in the builder chain — no struct-tag inference to fall back on, and no need to touch the handler at all for doc checks, which is genuinely simpler than the official SDK's style. Also found the tool's own name is commonly a package-level const reference (ToolConversationsHistory = "conversations_history") rather than an inline literal, resolved the same name-based, ambiguity-safe way as the struct/function registries already built. First run: 0→21 tools, 100%/A — verified by cross-checking several tools' resolved parameter counts against their real mcp.WithString/mcp.WithBoolean declarations in source by hand, the same discipline that caught two silent bugs in the previous pass. 6 new regression tests (83 total); re-verified no regression on xiaohongshu-mcp and chrome-devtools-mcp (the official-SDK style is a structurally separate code path — 3 AddTool arguments vs. 2 — so the two styles can't be confused with each other).

An eighteenth pass targeted the highest-star untested repos left in the "dedicated MCP server" pool (as opposed to huge generalist platforms that merely mention MCP support — checked and ruled several of those out first, including one, DeusData/codebase-memory-mcp, with a suspicious 42k-star/176-watcher ratio not worth trusting or citing regardless of what it would have found). containers/kubernetes-mcp-server (2k★, official Red Hat/containers repo, actively maintained) reported 0 of 39 tools — the largest gap by proportion since chrome-devtools-mcp. Root cause: this repo never calls mcp.NewTool(...) anywhere; every tool is fully literal data via a project-local api.ServerTool{Tool: api.Tool{Name: ..., Description: ..., InputSchema: &jsonschema.Schema{...}}, Handler: ...} struct — a shape one step more literal than hashicorp/terraform-mcp-server's server.ServerTool{Tool: mcp.NewTool(...), ...} (a builder call wrapped in the same struct), and in two forms: a lone api.ServerTool{...} composite literal, and — for files with multiple tools — a []api.ServerTool{...} slice whose elements elide the type name entirely (the same elision convention already handled for []*mcp.Tool{...}, just one level of struct nesting deeper). Extracted the shared "build a finding from a literal Tool + its handler" logic (previously only reachable from an AddTool call site) into its own helper so both the single-literal and slice-element paths reuse it, then reused the existing explicit-InputSchema param-doc check unchanged — no new schema-walking needed, since this repo's &jsonschema.Schema{Properties: map[string]*jsonschema.Schema{...}} shape is exactly what that check already resolves. Getting those 39 tools visible surfaced a second, independent, more generally useful bug: 5 of them build their description via fmt.Sprintf("...%s...", defaults.ProductName()), and _resolve_string_field's existing call-expression handling only ever checked the last argument — correct for the t(key, fallback) i18n convention it was written for, wrong for fmt.Sprintf, whose format string is always the first argument by the language's own contract, independent of whether the remaining args resolve. Fixed by special-casing fmt.Sprintf to resolve its first argument instead. Verified live: 0→39 tools, 97%/A. The 4 tools still correctly flagged (resources_list/get/create_or_update/delete) build their description from a local variable reassigned across conditional branches earlier in the same function — genuinely unresolvable without guessing at which branch ran, so left honestly flagged rather than forced. 4 new regression tests (132 total); re-verified all 7 previously-tested Go repos live with no regression (one, skyhook-io/radar, now shows 30 tools instead of the documented 31 — confirmed by re-running the pre-fix analyzer against the same current checkout and getting the identical 30, i.e. the repo itself changed since it was last dogfooded, not a regression from this fix).

The same round also checked four more of the highest-star untested dedicated servers, all clean or correctly-attributed to the target rather than mcp-doctor: Jpisnice/shadcn-ui-mcp-server (3k★, TS, 10/10 tools, 96%/A), financial-datasets/mcp-server (2.3k★, Python, 96%/A, one genuine doc gap on get_crypto_prices verified by hand against its bare one-line docstring), benborla/mcp-server-mysql (2.1k★, TS — correctly reports exactly 1 tool, confirmed against the repo's own README that it deliberately exposes one consolidated query tool), and GongRzhe/Office-Word-MCP-Server (2.1k★, Python, 91%/A — most of its 54 tools genuinely lack per-parameter docs, verified by hand against several, a real gap in the target worth a filed issue if revisited).

A nineteenth pass targeted two more official-vendor servers left in the untested pool: brave/brave-search-mcp-server (1.4k★, TS) and matlab/matlab-mcp-server (1.5k★, Go). Brave reported 7 of 8 tools with no description — a suspicious hit rate worth investigating (the same instinct that paid off on SurfSense and Windows-MCP), not trusting a clean-looking miss at face value. Every tool declares its description as a top-level export const description = \...`.trim()in its own file, then references it by identifier at theregisterTool call site — a real, common idiom for trimming an indented multi-line template literal, but the analyzer's identifier resolver (_resolve) had no case for a .trim()/.trimStart()/.trimEnd()call, so it fell through to "unresolvable" instead of resolving through to the untrimmed receiver (whitespace trimming never changes the content actually being checked for presence or length). Fixed the same way.filter()was already resolved through — a call that changes formatting or visibility, not identity. Verified against a minimal single-file repro first (passed, ruling out a cross-file name-collision theory), then reproduced the real 7-tool miss in isolation on one real tool file before fixing, then re-verified live on the full repo: 80%→100%/A. 6 of the 8 real descriptions are then correctly flagged "unusually long" (688–1396 chars) by the security check — read every one by hand: all genuinely detailed usage docs with real citation examples, no smuggled instructions, the same false-positive-on-real-detail pattern already documented forwigolo. 1 new regression test (133 total); re-verified chrome-devtools-mcp(61/93%A),mcp-server-cloudflare(143/93%A),mcp-server-browserbase(6/95%A),n8n-mcp(still 0 tools), andFigma-Context-MCP(2/89%B) all unchanged, including two security-score readings confirmed identical against the pre-fix analyzer to rule out an unrelated regression.matlab/matlab-mcp-server correctly reports 0 tools — genuinely out of scope, not a bug: every tool is built through a fully generic, reflection-based framework (ToolAdder[ToolInput, ToolOutput], jsonschema.ForToolInputinferring the schema at runtime via Go generics rather than any literal struct-tag call site) withname/title/descriptionas plain per-package identifiers and no literalmcp.Tool{...}/AddTool(...)call site anywhere in the tool's own package to anchor on — the same category as the already-declinedfastapi_mcp/XcodeBuildMCP`, correctly held off rather than building single-repo-specific support for one very particular factory-argument-position convention.

A twentieth pass revisited two already-audited repos through the security checks specifically — czlonkowski/n8n-mcp (23k★, security 0%/F despite the tool-detection limitation above) and MODSetter/SurfSense (16k★, already 98%/A on quality from the eleventh pass). n8n-mcp's F turned out to be almost entirely mcp-doctor's own false positives, not a real security gap: a TS method literally named exec (a DatabaseAdapter.exec(sql): void interface delegating to a SQL driver's own safe .exec()) matched the dangerous-call pattern — a declaration-vs-call confusion the existing .exec( member-call exclusion didn't cover — and raw-text scanning matched the literal words "eval("/"exec(" inside n8n-mcp's own eval/exec-detecting validator code, both its user-facing warning strings ('Avoid eval() - it's a security risk') and a comment illustrating what it checks for, an especially ironic case since that file explicitly strips strings/comments before scanning for exactly this reason and mcp-doctor didn't do the equivalent for itself. Fixed both (a declaration-shape exclusion; string-literal and //-comment masking before matching). Re-verified: security 0%→70%/C, remaining findings all the already-documented conservative SSRF heuristic (one with an explicit GHSA advisory reference and pinned-transport mitigation already in place). Checking SurfSense's security findings the same way surfaced a third, related false positive: token_quota_service.py's await r.eval(ACQUIRE_STREAM_LUA, 1, key, ...) — a Redis client's EVAL command running a fixed Lua script server-side — was flagged the same way JS's dangerous eval() builtin would be, since the existing eval exclusion only covered Playwright's $eval(), not general dot-preceded member calls the way .exec( already was. Generalized the exclusion, but added back an explicit window.eval()/globalThis.eval() pattern so the real bare-eval-detection bypass — dot-preceded, but still the genuine builtin — stays caught. Re-verified: SurfSense dangerous_exec 9→2 findings (both real subprocess.run() calls, not false positives), security 96%→97%/A; n8n-mcp re-confirmed unchanged by this second fix. 5 new regression tests (141 total).

A twenty-first pass revisited oraios/serena (29.2k★, already dogfooded once for the class-based-tool registry itself) looking for a second, independently-sourced target — microsoft/playwright-mcp (37k★) and googleapis/mcp-toolbox (16k★) were both tried first and correctly ruled out: the former's actual tool code lives entirely in the upstream microsoft/playwright monorepo, not this repo, and the latter builds every tool dynamically from YAML config through a generic Tool interface, with no literal call site anywhere to anchor on — the same "no anchor point" category as the already-declined matlab-mcp-server/fastapi_mcp. Re-scanning serena found a real, if narrow, false positive: search_for_pattern and safe_delete_symbol were flagged "Tool has no description. An agent cannot decide when to call this" — an ERROR-level finding, the same severity as a missing decorator. Both tools are real and well-documented; the bug was in _find_class_based_tools itself, which always read the tool's description from the class's own docstring (ast.get_docstring(node)) and passed it as description_override unconditionally — even when that docstring was empty — so _analyze_function_as_tool's existing fallback to the function's own docstring (used everywhere else in the file) never got a chance to fire for this one registration style. Read serena's own src/serena/mcp.py before assuming anything: func_doc = tool.get_apply_docstring() or "" is what actually gets passed as description= at tool-registration time — the apply() method's docstring, never the class's — and tools_base.py documents this explicitly ("The docstring and types of the apply method are used to generate the tool description"). Most of serena's 43 tools happen to carry a docstring on both the class and apply(), which is why only 2 of them ever surfaced the gap. Fixed by passing None instead of "" when there's no class docstring, letting the existing fallback do its job — a one-line, narrowly-scoped fix. Caught and fixed a second bug in the same pass: an existing regression test asserted the old, now-disproven behavior ("apply()'s own docstring must not be substituted in as a fallback") based on an assumption about serena's runtime behavior that was never actually checked against mcp.py — replaced it with two tests, one confirming the correct fallback and one confirming a tool with no docstring anywhere (neither class nor apply()) still correctly fails. Re-verified: 96%→97%/A, 142 tests total, no regression on any other repo (this code path is serena-specific — no other dogfooded repo uses the class-based registry). Also ran mcp-fuzz and mcp-reality-check against serena end-to-end for the first time, with a real Python language server actually running (uv/uvx installed for this, then 28/28 tools launched via --include-destructive, including every symbol/LSP-backed tool) — 0 crashes, 0 timeouts, 100%/A on both; the deepest real-server coverage any single repo has gotten in this project so far.

A twenty-second pass against sooperset/mcp-atlassian (5.9k★, Python, the widely-used community MCP server for Jira/Confluence) reported 2 tool names declared more than once (add_comment, search) — a spec violation ("SHOULD be unique within a server") on a repo with no obvious reason to duplicate anything. Both names are real: each is defined once in servers/jira.py and once in servers/confluence.py, and the repo composes them into one server via FastMCP's main_mcp.mount(jira_mcp, namespace="jira") / main_mcp.mount(confluence_mcp, namespace="confluence") — a documented FastMCP pattern (confirmed against fastmcp's own mount() docstring) that renames every tool on a mounted sub-server to namespace_toolname at the protocol level. So the two search tools are actually jira_search/confluence_search to a real client — not duplicates at all; mcp-doctor's uniqueness check just didn't know mounting existed. Fixed by resolving X.mount(Y, namespace="ns") calls repo-wide (name-based, same simplification as the existing alias/error-handling registries — the mount target must be a simple name assigned via exactly one name = SomeCall(...) in the whole repo, ambiguous cases dropped rather than guessed) and applying the namespace prefix to every tool found in the resolved file before the dedup check runs. Re-verified: the false duplicate warning is gone, and the 98 real tools are now correctly named jira_search/confluence_search/jira_add_comment/confluence_add_comment etc. throughout — including in the README-coverage check, which is now checking tools against their real client-facing names. 1 new regression test (162 total); no change on any other repo (only mcp-atlassian uses .mount(..., namespace=...) in the dogfooded set so far). Final result: 98 tools, 100%/A quality, 99%/A security — the 3 remaining security findings (oauth.py:177/277/322) are the already-documented conservative SSRF heuristic, verified by hand: all three flag requests.post(token_endpoint, ...) where token_endpoint is the OAuth provider's own fixed token URL, not a tool-argument-controlled value. The one real, uncorrected quality gap: 88 of 98 tools aren't mentioned in the README (a large, genuinely under-documented public surface, not a false positive).

A twenty-third pass against jingcheng-chen/rhinomcp (1k★, Python, connects Rhino 3D CAD to an AI agent) found a second, more general variant of the same class of tool-name-uniqueness false positive: 10 tool names flagged as duplicates, none of them real. This repo ships one real production server (server/src/rhinomcp/, 70 tools) alongside dozens of independent scratch/experimental servers under experiments/ (each its own if __name__ == "__main__": mcp.run() entrypoint, several reimplementing a tool like create_object or analyze_objects under the same name purely for local testing). The spec's "SHOULD be unique within a server" doesn't apply across two servers that can never both be running — but the existing uniqueness check compared every tool found anywhere in the repo against every other, with no notion of which file belongs to which running server. Fixed by detecting standalone if __name__ == "__main__": x.run() entrypoint files repo-wide (excluding anything already resolved as a .mount() target from the twenty-second pass, since composed-but-also-testable files should stay grouped with what they're mounted into) and scoping the duplicate check to compare only within each such file, not across them. Re-verified: 9 of 10 false duplicates cleared. The tenth (join_surfaces) is a genuinely harder case — experiments/integration_mcp.py does mcp = visual.mcp, literally reusing another experimental file's FastMCP instance rather than constructing its own — correctly left flagged rather than building cross-file instance-alias resolution for one remaining warning in non-shipped scratch code. 2 new regression tests (164 total); re-verified mcp-atlassian unchanged (100%/A quality, 99%/A security, tool_name check clean). Final result: 93 tools, 93%/A quality, 98%/A security.

A twenty-second pass added a new security check rather than another fix: a real, verified precedent (DeusData/codebase-memory-mcp#2118, filed by this same project — 13 of 15 tools mislabeled destructive/read-only) shows a mismatched readOnlyHint annotation is a real, recurring bug class, not hypothetical — an agent framework that gates approval prompts on that hint can skip confirming an action that isn't actually read-only. mcp-doctor already had the annotation-respecting half of this (mcp-fuzz does, at runtime); this adds the annotation-verifying half: for a Python/FastMCP @mcp.tool(annotations=ToolAnnotations(read_only_hint=True))-decorated tool, statically scan the tool's own function body (ast.unparse(fn), reusing the AST node already in hand rather than needing the original source text) for a real write/mutation signature — a raw SQL mutation verb, a file opened in write/append mode, a filesystem deletion call, or a mutating call on a recognizable HTTP client. Deliberately narrow: .save()/.commit() alone are common enough on genuinely read-only code paths (an ORM's read-only session, a result saved to a local variable) to be more noise than signal, so they're not included. 5 new regression tests (147 total): a file-write and a raw-SQL-mutation case both correctly flagged, a clean read-only tool and an undeclared (no annotation either way) mutating tool both correctly left alone, and confirmation the new issue lands in the security score, not quality. Verified against three real, previously-untested Python repos with genuine readOnlyHint usage in real (non-test) source — CursorTouch/Windows-MCP, atilaahmettaner/tradingview-mcp, and a new one, jingcheng-chen/rhinomcp (1k★, 93 tools) — all three clean, 0 false positives, no bug found this round. An honest "nothing yet" on real targets is itself worth stating rather than only reporting passes that found something.

A twenty-third pass added a feature this project's own history had already surfaced a real need for, rather than another dogfood target: a schema/contract diff mode. Every check so far scores one snapshot; nothing here caught a change between two runs — the exact class of gap Tyler Robinson (a real commenter on the trilogy LinkedIn article) named directly: "permission scoping, contract consistency, or observability gaps across tool boundaries." Added --diff-against PATH, comparing the current run's per-tool parameter signatures against a previously saved --json report: a tool that's gone, a parameter that's gone, or a parameter that's newly required (either brand new, or previously optional) are all flagged as breaking; a new tool or a new optional parameter — nothing an existing caller relied on stopped working — is correctly never flagged. Required adding param_names/required_param_names to ToolFinding itself (previously only counts were tracked, e.g. param_count, not the actual names needed to diff two runs against each other) — populated in _analyze_function_as_tool, which every Python registration style already funnels through (FastMCP decorator, resolved Tool() direct calls, and serena's class-based registry all get it for free), deliberately left at the pre-existing empty default for the raw-schema-dict Tool(name=..., inputSchema={...}) constructor style and for TS/Go, the same incremental, language-by-language and pattern-by-pattern scoping already used throughout this analyzer. 12 new regression tests (159 total). Verified two ways: a synthetic before/after pair confirming a removed parameter is caught and --fail-on-breaking-change exits non-zero; and a real check against jingcheng-chen/rhinomcp's own git history — checked out a commit 20 revisions behind main, scanned it, checked back out to main, and diffed — honestly clean (many tools added since, none removed, no parameter newly required), a real answer from real history rather than a synthetic-only proof.

A twenty-fourth pass extended the public leaderboard rather than the CLI itself: drift tracking across scans. Every prior run silently overwrote the last one — a repo's grade could regress and nothing would show it. Fixed without adding any new committed state: leaderboard/scan.py now fetches whatever's currently live at the start of each run and diffs the new scan against it, before overwriting it — the deployed data.json already is "the last scan," so there's no need for CI to commit a separate history file back to the repo. Each row gains quality_percent_change/security_percent_change (None, not 0, when there's nothing to compare against yet — a brand-new repo's first scan isn't "unchanged"), rendered on the live page as a real ▲/▼ delta next to the grade. Caught something real while verifying it, worth stating plainly: an initial local check showed homeassistant-ai/ha-mcp "regressing" 99%→98% between two scans of the identical git commit — not a bug in the drift logic itself, but a mismatch between the Python version running locally (3.11) and the one the Leaderboard workflow actually runs in CI (3.12), surfacing the already-documented "parses with the running interpreter's grammar" limitation from a new angle. Confirmed by installing Python 3.12 locally and re-running: identical commit, identical score, 0 drift — real production comparisons are always CI-scan-vs-CI-scan (3.12 vs 3.12), so this specific noise source doesn't reach the deployed page, but it's a genuine caveat worth naming for anyone re-running the scan script locally with a different Python than CI's. Re-ran the full 19-repo leaderboard scan under matching Python 3.12 afterward: 0 drift across every repo, the correct, honest result for two back-to-back scans of unchanged targets. Verified the frontend rendering visually in a real browser (not just reading the generated HTML) with synthetic test deltas before shipping the real, clean data.

A twenty-fifth pass ran the full, current toolkit — every check across all three tools — against one fresh target at once for the first time, rather than verifying each check individually against a different repo the way every prior pass had. sooperset/mcp-atlassian (5.9k★, real Jira/Confluence integration, actively maintained) was picked specifically for having genuine ToolAnnotations usage in source, real enough to exercise the annotation-mismatch check meaningfully. mcp-doctor's own result: 98 tools, 100%/A quality, 99%/A security, --diff-against a commit 30 revisions behind main honestly clean. One already-known false-positive class re-confirmed rather than newly found: search/add_comment flagged as duplicate names, verified by hand to be two separate tools in two separate sub-servers (jira.py/confluence.py) that happen to share a name — exactly the documented "no call-graph awareness" limitation, not a new gap. The more interesting result came from a sibling tool, not this one: getting mcp-atlassian running at all required dummy Jira/Confluence credentials passed through as environment variables, and doing that surfaced a real, verified bug in mcp-reality-check — it had no --env flag at all, despite mcp-fuzz solving the identical problem months earlier for brave-search-mcp-server. See mcp-reality-check's own README for that fix — flagged here too since it's the reason this pass could complete at all once found.

A twenty-sixth pass added a new security check rather than another dogfood target: unpinned dependencies, the one genuinely open item left after a full audit of what a complete Agentic AI evaluation framework would need (drift tracking, schema-diff, and concurrency had already closed the others). Deliberately scoped narrow — not a real vulnerability-database scan (that means a network call and an external tool dependency like pip-audit/npm audit, a real architecture change to a tool that's been fully offline and dependency-free since day one), but the same pure-text approach as everything else here: a requirements.txt line with no version constraint at all, or a package.json dependency pinned to */latest, flagged as a real, if narrow, supply-chain signal — a compromised or typosquatted release of that package gets pulled in automatically at install time, no code change in this repo required. The false-positive-rate judgment mattered as much as the detection: an npm caret/tilde range (^4.18.0) or a Python >=/~= floor is the default output of npm install --save and completely standard practice — flagging every one would be far more noise than signal, the same call already made for the annotation-mismatch check's .save()/.commit() exclusion. Go's go.mod and pyproject.toml's several possible dependency-table shapes are correctly left out rather than guessed at. Building this immediately surfaced a real, if narrow, bug in the test suite itself: the shared CLEAN_FILES fixture's own requirements.txt ("mcp\n", no version at all) had never actually been clean by this new definition, and once the check could see it, two existing tests asserting a "clean repo" scores 100% security broke — fixed at the source (pinned the fixture) rather than special-cased around. 9 new tests (161 total). Verified against six real, already-known repos rather than a synthetic-only proof — homeassistant-ai/ha-mcp, sooperset/mcp-atlassian, oraios/serena, jingcheng-chen/rhinomcp, CursorTouch/Windows-MCP, atilaahmettaner/tradingview-mcp — all six genuinely clean, 0 false positives, no finding this round; an honest "nothing yet" on real, well-maintained targets, same discipline as the annotation-mismatch check's own first verification round.

A twenty-seventh pass fixed a real false positive in --diff-against, reported externally rather than found by dogfooding: Edward Izgorodin found that diffing a baseline captured from a FastMCP-decorated function against a current run captured from the raw Tool(inputSchema=...) constructor style reported a shared, unchanged, required parameter as param_removed — because the raw-schema style never populated param_names/required_param_names at all, so "genuinely no parameters" and "coverage not attempted" were indistinguishable, and the comparison read the latter as the former. His minimal before/after repro (two lookup(key: str) tools, one per style) reproduced it exactly. Fixed at the source rather than papered over with a coverage flag: _find_lowlevel_tools now statically resolves param_names from the schema's properties keys and required_param_names from its required list, the same real information the raw-schema style always had available but never extracted — closing the gap outright instead of just suppressing the false positive. Kept honest about what's still unresolvable: if any property name in the schema isn't a string literal (a dynamic key, or an unresolved **spread_call()), param_names is incomplete, so required_param_names is deliberately dropped back to empty for that tool rather than risk misreading a still-present property as newly required. 4 new regression tests (167 total) — Edward's exact repro as a no-op, a same-style-would-have-caught-it real-removal case, and the dynamic-key fallback. TS/Go tools are still correctly out of scope, per the limitation below.