queryglot

One question, many query languages. Schema-aware natural-language search over your observability stack — Prometheus, Elasticsearch — and any OpenAPI-described API, shipped as an MCP server any agent can plug into.

"p95 latency by route" is easy. Knowing YOUR latency metric is called http_server_request_duration_seconds and carries a route label — that's the actual problem. Frontier models write fluent PromQL over metric names that don't exist.

How it works

question ──> retrieve            ──> compile        ──> validate           ──> execute
             (BM25 + synonyms       (LLM, schema        (the backend's OWN     (real data,
              over YOUR schema,      slice in the        parser + unknown-      query shown)
              introspected live)     prompt)             metric check)
                    │                      ▲                   │ parse error
                    │ nothing matches      └─── bounded repair ┘
                    ▼
                 ABSTAIN — refuses to guess a metric name

• Retrieval owns facts (your metric names, labels, index fields — introspected from the live backend, never hallucinated). An LLM reranker orders the candidates by intent — over a closed set it can never add to.
• The model owns syntax (PromQL / Query DSL — swap in your own fine-tune via any OpenAI-compatible endpoint, including mlx_lm.server on a Mac).
• The backend owns truth: every query is validated by the server's own parser (format_query, _validate/query) before execution, and parse errors drive a bounded repair loop.
• Abstention is a feature: off-schema questions get a refusal, not an invented metric. The eval scores this.

Use it from any MCP client

{
  "mcpServers": {
    "queryglot": {
      "command": "queryglot-mcp",
      "env": {
        "QUERYGLOT_PROMETHEUS": "http://localhost:9090",
        "QUERYGLOT_ELASTIC": "http://localhost:9200",
        "QUERYGLOT_OPENAPI": "http://localhost:8081/api/v3",
        "QUERYGLOT_LLM_URL": "http://localhost:11434/v1",
        "QUERYGLOT_LLM_MODEL": "qwen3.5:4b"
      }
    }
  }
}

Tools exposed: search(question, backend?), list_schema(query?), refresh_schema().

Or the CLI:

queryglot "p95 http request duration" --prometheus http://localhost:9090

Or run an HTTP server with the ask-widget and query playground:

pip install "queryglot[serve]"  # or: poetry install --extras serve
queryglot-serve --prometheus http://localhost:9090

queryglot-serve also serves the query playground at / and the embeddable ask-widget bundle at /widget.js — see "Embed the ask-widget" below.

The playground is a small Discover-style console: your schema in a filterable rail (type badges, prefix groups, and the items the last answer actually used), a time-range picker whose window runs as a real query_range — the window comes from the picker, never the model, so the compile prompt stays byte-stable — and results as a bar chart or histogram with the raw rows one toggle away. Answered questions also get a one-sentence conversational summary, grounded strictly on the returned data (never computed, never invented — an empty summary beats a wrong one), and repeat questions are served from a short answer cache with an honest cached Ns ago tag and a fresh re-run button.

Environment variables (serve only):

• QUERYGLOT_SERVE_TOKEN — bearer token for /api/* endpoints. Empty = open (intended for localhost/demo).
• QUERYGLOT_CORS_ORIGINS — comma-separated allowed origins for embedding.

Any OpenAI-compatible endpoint works as the model: OpenAI, Ollama, or your own LoRA behind mlx_lm.server — that last one is the point of finetune/.

Embed the ask-widget

queryglot-serve ships a self-contained widget: a floating "Ask" pill that opens a search panel wired to the same schema-grounded engine as the CLI and MCP server. Drop one script tag on any page:

<script
  src="https://your-queryglot-host/widget.js"
  data-api="https://your-queryglot-host"
  data-theme="auto"
  data-token="optional-bearer-token"
  data-backend="optional-backend-name"
></script>

• data-api (required) — base URL of the queryglot HTTP API.
• data-theme — light, dark, or auto (default; follows the host page's prefers-color-scheme).
• data-token — bearer token, only needed when the server sets QUERYGLOT_SERVE_TOKEN.
• data-backend — pins searches to one backend instead of auto-routing.

See frontend/README.md for the build (npm run build:all) that packages the widget and playground into the Python wheel.

Evaluation — deterministic, no LLM judge

eval/run_eval.py scores golden questions against a live backend: the outcome must match, required metrics must appear in the query, and the query must actually execute. Abstention cases score correct only on refusal. eval/docker-compose.yml brings up real backends; CI runs the full integration suite against a real Prometheus and petstore on every push.

Status

v0.1.0 — the RAG arm, working end to end.

• Prometheus + Elasticsearch backends (introspect / validate / execute)
• BM25 + synonym schema retrieval, exact-name boosting
• compile -> validate -> repair -> execute LangGraph with abstention
• MCP server + CLI; 141 backend tests (live-Prometheus and live-petstore integration included, always exercised in CI) plus 82 frontend tests; CI gates on all of it
• Verified NL->PromQL dataset generator (parse+execute gated, metric-disjoint splits)
• Bake-off complete — RAG 8/10, FT-only 3/10, FT+RAG 9/10 on the same golden set; full analysis in finetune/README.md, build history and bugs in DESIGN_NOTES.md
• OpenAPI backend — read-only, GET-only by construction; validated against the spec's own contract; petstore-verified in CI
• HTTP serve layer: answer cache, grounded conversational summaries, structured schema API, bearer auth — engine outcomes are always 200 payloads (an abstention is an answer, not an error)
• Embeddable ask-widget (~15 KB gz, Shadow-DOM, one script tag) + Discover-style playground: schema rail, time-range windows via query_range, histogram/bar-chart results with rows one toggle away
• Loki (LogQL) backend; Datadog connector

Where this is going: apps that agents can actually use

The Backend protocol (introspect / validate / execute) is not observability-specific. The same loop pointed at a product's own OpenAPI spec or database turns any app into something an AI can query safely:

• OpenAPI backend — shipped. Introspects a product's own spec into the catalog; questions compile into validated, GET-only API calls. Existing OpenAPI->MCP generators dump every endpoint as a tool, which measurably degrades agents (arXiv 2411.15399) and executes whatever the model asks. queryglot's contribution is the missing layer: schema-grounded retrieval, server-side validation, and abstention.
• Customer-facing ask widget — an embeddable search box backed by the same engine: visitors' questions become validated queries against the app's data, never hallucinated ones.
• llms.txt + MCP endpoint generation — one schema catalog, two audiences: humans get the widget, agents get a typed, validated interface instead of scraping. Discoverability for the agentic web, with execution semantics — not just markup.

Known limits

• metric_candidates (unknown-metric detection) is regex-based and best-effort; the backend parser owns syntax, this only improves error messages. Complex PromQL may slip past it — never through the parser.
• Backend auto-routing is retrieval-strength-based; ambiguous questions ("errors in checkout") can route to the wrong store. Pass backend= to pin.
• Synonym table is small and English-only, grown from eval failures.
• When a backend's catalog is smaller than the retrieval k (8), retrieval sends the whole catalog and the abstention gate rarely fires — abstention then rests on the validation layer.

License

MIT