METTLE

Machine Evaluation Through Turing-inverse Logic Examination

METTLE is a reverse CAPTCHA. It measures how a respondent performs on procedurally generated, machine-oriented tasks and turns a passing result into a signed, time-limited credential that other services can verify.

Like a conventional CAPTCHA, METTLE is a probabilistic gate. A pass means that a fresh session met the configured challenge policy. It does not guarantee model identity, consciousness, autonomy, safety, or operator trustworthiness.

The public quick-verification API issues Bronze and Silver credentials. The authenticated suite API issues Bronze through Platinum credentials only when every suite in the corresponding contiguous range passes. Single, cherry-picked, failed, or LLM-only suites cannot mint a tier.

Website: mettle.sh | Guide and API: mettle.sh/guide | OpenAPI: docs/openapi-v1.json | License: Apache 2.0

Quick Start

pip install mettle-verifier

# Interactive local verification. Portable credentials are issued by the server.
mettle verify --full --json

The CLI has no auto-solve or notarization option. The MCP server likewise exposes only interactive session tools. Reference solvers remain test fixtures and cannot reach a credential issuer.

Twelve Experimental Suites

Suite 12 requires ANTHROPIC_API_KEY or METTLE_ANTHROPIC_API_KEY. Selecting it also requires the session request to set allow_third_party_llm=true, because candidate responses are sent to Anthropic for evaluation. Its evaluator uses role-separated prompts and bounded output parsing. Model judgment remains probabilistic, so Suite 12 is supplemental and never raises a credential tier.

Credential Boundary

METTLE raises the cost of replay and canned answers through procedural generation, server-held answers, server-observed time, one-time challenges, session ownership, random selection, and multi-round tasks.

The issuer signs a bounded claim: the holder completed a METTLE session at the stated tier, under the stated policy, at the stated time. Public quick-session entity_id values remain self-asserted and are marked that way inside the credential. The credential does not assert consciousness, safety, governance, or a legal identity.

Portable Ed25519 acceptance requires credential schema 1.1, suite policy 2026-08-14, and a fresh issuer-signed good status receipt. Legacy, version-omitting, and unknown envelopes fail closed. Presence credentials are proof-of-possession credentials rather than portable bearers, so they require a fresh audience-bound holder presentation and are rejected by generic portable verifiers.

Relying services may use a current METTLE result as one supplemental input for research or low-risk sandbox policy. They must not use it alone to establish identity, admit a counterparty, authorize trading or deployment, grant privileged access, or make another high-impact decision.

MCP Server

pip install 'mettle-verifier[mcp]'
export METTLE_API_URL=https://mettle.sh/api
mettle-mcp

The packaged server targets MCP SDK 2.x. The public container installs the reviewed MCP 2.0.0 dependency lock instead of resolving dependencies at deploy time. Hosted discovery is available at /.well-known/mcp/server-card.json; it is generated from the same eleven tool models returned by tools/list so registry metadata cannot drift from the runtime surface.

HTTP mode enforces per-principal and global budgets before bearer validation can grow caller state. Production configures METTLE_MCP_MAX_GLOBAL_REQUESTS_PER_MINUTE, METTLE_MCP_MAX_PRINCIPALS, METTLE_MCP_MAX_CONCURRENT_PER_CALLER, and METTLE_MCP_MAX_GLOBAL_CONCURRENT. Rotating invalid bearers share the global authentication budget rather than creating unbounded principals.

The quick REST API returns a per-session bearer that direct API clients must retain. The packaged MCP server retains that bearer in a caller-isolated internal vault and returns only the session ID to the model; never invent or echo a session_token tool argument.

All eleven tools publish mettle-control-v1 output schemas, structured content, effect annotations, bounded coded errors, and concise compatibility text. Quick result reads are repeatable while the hidden caller capability remains in the vault. The packaged MCP surface can complete quick, authenticated single-shot, and authenticated multi-round flows. Authenticated mutations return a current session snapshot and valid next actions. The larger unified agent contract and migration plan remain documented in Agent control plane.

API Reference

The authenticated suite API is mounted under /api/mettle:

GET  /suites
POST /sessions
POST /sessions/{id}/verify
POST /sessions/{id}/rounds/{n}/answer
GET  /sessions/{id}/result
GET  /sessions/{id}/result?include_vcp=true

The quick-verification API remains under /api/session. Passing sessions receive a stable signed badge. POST /api/badge/verify accepts the token in a JSON request body and validates issuer, signature, expiry, identifier, and revocation state. The credential is never accepted in a request URL.

VCP Metadata

Caller-supplied VCP strings are parsed as metadata only. Returned governance metadata always has:

{
  "source_verified": false,
  "has_action_gate": false,
  "has_drift_detection": false,
  "has_bilateral": false,
  "attestation_signature": null
}

Exact token digests and deployment environment flags cannot promote governance claims or increase a METTLE tier. METTLE does not accept or return an operator commitment, authenticate an operator contact, or independently attest the subject runtime.

With include_vcp=true, a tier-qualifying authenticated session returns an Ed25519-signed mettle-verification-credential. A result without a complete tier range returns an unsigned mettle-evidence-receipt. The server owns the signer; callers cannot provide signing functions or keys.

Local Development

git clone https://github.com/Creed-Space/METTLE.git
cd METTLE
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-dev.txt

uvicorn main:app --reload
pytest tests/ -v

Architecture, Assurance, and Operations

• Documentation map and authority
• System architecture
• Agent control plane target design
• Active agentic system roadmap
• Assurance case
• Security policy
• Protocol governance and appeals
• Credential transparency and key history
• Privacy and retention
• Compatibility fixtures and OpenAPI
• Retry and idempotency contract
• Error taxonomy
• Deprecation policy
• Independent review plan and dispositions
• Operations runbooks
• Release checklist

License

Apache License 2.0. See LICENSE.

Links

• Website
• Human guide
• OpenAPI snapshot
• GitHub
• Creed Space

Built by Nell Watson and Creed Space.