woodstitch-mcp

An MCP server for the Woodstitch API: paid human listeners judge your text-to-speech audio in blinded studies, and your agent gets a number back with a confidence interval.

Two tasks, and only two: pairwise preference (a listener hears two clips of the same text and picks one, forced choice) and human-vs-AI detection (one clip, human or AI, confidence 1–4). Not offered: MOS, CMOS, MUSHRA or any 1–5 scale.

Install

Get an API key at https://woodstitch.com/app/api-keys, then:

Claude Code

claude mcp add woodstitch -e WOODSTITCH_API_KEY=ws_live_... -- uvx woodstitch-mcp

Claude Desktop / Cursor / any MCP client (mcpServers entry):

{
  "woodstitch": {
    "command": "uvx",
    "args": ["woodstitch-mcp"],
    "env": {"WOODSTITCH_API_KEY": "ws_live_..."}
  }
}

pipx run woodstitch-mcp or pip install woodstitch-mcp && woodstitch-mcp work too. Requires Python 3.10+.

Claude Code users: the Woodstitch plugin bundles this server with a skill that knows when and how to run a listening test: /plugin marketplace add woodstitch-llc/claude-plugin then /plugin install tts-human-eval@woodstitch.

Tools

Safeguards built in

• woodstitch_start_eval requires user_confirmed_amount_usd, the quoted total the user agreed to, and sends it to the API as the hard ceiling max_total_usd. If the quote is higher than what the user confirmed, the API refuses (over_max) and nothing starts. An agent cannot start a study the user has not seen the price of.
• Every POST carries an Idempotency-Key, so a retried call never creates or starts a study twice.
• The server's instructions tell the model that interim numbers are provisional and that stopping early on a threshold is optional stopping.

Typical session

1. woodstitch_upload_files(["a/001.wav", "b/001.wav", ...]) or use public URLs.
2. woodstitch_create_eval(pairs=[{"id":"001","a":"file_…","b":"file_…","passage":"001","system_a":"base","system_b":"candidate"}, …], ratings_per_pair=5) → quote, nothing charged.
3. Show the user the total. On yes: woodstitch_start_eval(eval_id, user_confirmed_amount_usd=<total>).
4. woodstitch_get_eval(eval_id) now and then; first judgements usually land within minutes.
5. woodstitch_get_results(eval_id) when status is completed; woodstitch_export for the rows.

Method, listener pay and controls: https://woodstitch.com/research/methodology. Machine-readable API spec: https://woodstitch.com/openapi.json.

Development

uv venv && . .venv/bin/activate && uv pip install -e .
WOODSTITCH_API_KEY=ws_live_... woodstitch-mcp        # stdio server

MIT © Woodstitch LLC