gc-eval-mcp 1.0.0

MCP server for BizChat Gov Cloud quality evaluation data

GC-Eval MCP Server

A manifest-driven MCP server that exposes BizChat Gov Cloud quality evaluation data to any MCP-compatible client (e.g., Copilot CLI).

No repo access required. Install once, always up-to-date — data is fetched from a hosted endpoint at runtime.

Quick Start (for consumers)

pip install gc-eval-mcp

Add to .copilot/mcp.json in any project:

{
  "mcpServers": {
    "GC-Eval": {
      "command": "gc-eval-mcp"
    }
  }
}

Done. Restart Copilot CLI and the tools are available.

How It Works

┌─────────────────────────────────────┐
│  Hosted Data (GitHub Pages / Blob)  │
│  manifest.json → describes datasets │
│  ├── HTML reports                   │
│  ├── JSON files                     │
│  ├── Markdown docs                  │
│  └── SQLite databases               │
└──────────────┬──────────────────────┘
               │ fetched at runtime
┌──────────────▼──────────────────────┐
│  GC-Eval MCP Server (local)         │
│  Reads manifest → registers tools   │
│  Caches data locally (~/.cache/)    │
└──────────────┬──────────────────────┘
               │ MCP protocol
┌──────────────▼──────────────────────┐
│  Copilot CLI / Any MCP Client       │
└─────────────────────────────────────┘

Available Tools

Always available (static)

Tool	Description
get_status	Server status, manifest info, errors
list_datasets	All available datasets
refresh_cache	Clear cache and re-fetch manifest

Registered per dataset (dynamic, from manifest)

Dataset Type	Tools Generated
html-collection	list_{prefix}, get_{prefix}_report, get_{prefix}_certification, search_{prefix}
markdown	get_{prefix}
json	get_{prefix}, query_{prefix}
sqlite	query_{prefix}, describe_{prefix}

Default tools (from example manifest)

Tool	Description
list_metrics	List all 23 scorecard metrics with certification
get_metrics_report(item_id)	Full HTML research report for a metric
get_metrics_certification(item_id)	Certification details (L1/L2/L3)
search_metrics(query)	Search metrics by ID or metadata
get_scorecard	Gov Cloud Scorecard markdown
get_certifications	Full certifications JSON
query_certifications(key_path)	Query certifications by dot-path

Configuration

Env Variable	Default	Description
GC_EVAL_MANIFEST_URL	https://gim-home.github.io/gc-eval-data/manifest.json	Where to fetch the manifest
GC_EVAL_CACHE_DIR	~/.cache/gc-eval-mcp	Local cache directory
GC_EVAL_CACHE_TTL	3600 (1 hour)	Cache TTL in seconds
GC_EVAL_MAX_DOWNLOAD_MB	50	Max file download size

Adding New Data (for maintainers)

To expose new data through the MCP, update the manifest — no server code changes needed:

{
  "id": "my-new-data",
  "type": "sqlite",
  "tool_prefix": "my_data",
  "path": "data/my_new_data.db",
  "description": "My new dataset"
}

Supported types: html-collection, markdown, json, sqlite.

Upload the data file to the hosted endpoint, add the entry to manifest.json, and all MCP users instantly have access.

Development

cd mcp-server
pip install -e .
gc-eval-mcp

Or run directly:

python -m gc_eval_mcp

Manifest Schema

See manifest.json for the full example. Key fields:

{
  "schema_version": "1.0",
  "data_version": "2026-05-07",
  "base_url": "https://your-host.com/data/",
  "datasets": [
    {
      "id": "unique-id",
      "type": "html-collection|markdown|json|sqlite",
      "tool_prefix": "valid_python_identifier",
      "path": "relative/path/to/data",
      "index": "optional/index.json",
      "description": "Human-readable description"
    }
  ]
}