bigfix-remote-client-relevance

Evaluate BigFix client relevance on remote endpoints and inside containers via qna, without a full BES install — over SSH, in Docker, or locally.

The point is a fast edit → evaluate loop while authoring content, instead of the minutes-long action-deployment round trip. A future MCP server can import this package so AI agents can write and test client relevance the same way.

Client relevance, not session relevance. This deals only with the dialect qna and the BES client evaluate on an endpoint. Session relevance — the bes-* object model queried through the root server's REST /api/query — is a different dialect and out of scope.

See DESIGN.md for the full design and rationale.

Install

uv tool install bigfix-remote-client-relevance

Or run it without installing:

uvx bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 "name of operating system"

Use

Evaluate on a container, provisioning a pinned qna version on the fly — no BigFix install, no SSH credentials, nothing to clean up:

bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 "name of operating system"

Compare the same expression across two qna versions on one target:

bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 --qna-version 10.0 "version of client"

Evaluate against the BigFix client on this machine:

bigfix-remote-client-relevance --local "name of operating system"

On macOS qna needs root, so --local implies --become there automatically — just the qna invocation runs under sudo -n, not the whole CLI. Pass --no-become to get the plain "needs root" refusal instead.

Evaluate on a real endpoint over SSH (a ~/.ssh/config alias works):

bigfix-remote-client-relevance mac-test --become "name of operating system"

Fan out across an inventory and emit JSON:

bigfix-remote-client-relevance --inventory hosts.toml -f probe.rel --json

--json writes one document per (target × version) to stdout; logs go to stderr, so piping into jq always works.

If no target is given at all — no --local, --container, --inventory, or HOST — and a hosts.toml exists in the current directory, it's used automatically, so the above also works as:

bigfix-remote-client-relevance -f probe.rel --json

Streaming

Results are emitted as each target answers, in completion order rather than inventory order — a slow SSH endpoint no longer holds up the containers that already finished. This applies to plain text and to --jsonl, which writes one compact JSON object per line:

bigfix-remote-client-relevance --inventory hosts.toml --jsonl "version of client" | jq -c '{host, elapsed_ms}'

{"host":"local","elapsed_ms":928}
{"host":"container:debian:12@x86_64","elapsed_ms":653}
{"host":"ssh:192.168.4.115","elapsed_ms":2603}

--jsonl carries exactly the same fields as --json; the only difference is the framing, so pick --jsonl for a line-oriented reader and --json when you want one document to parse in full. They are mutually exclusive.

--json and --diff are whole-set views — one array, and a grouping that only exists once every answer is in — so those two still print once at the end. Exit codes are always decided after the full fan-out, streaming or not.

Exit codes

Actionable for CI gating; the worst across the fan-out wins.

64 is sysexits EX_USAGE, deliberately outside the fan-out's range so a caller that shells out can tell a bad invocation from a failed evaluation. (It was 2 through 0.1.2, which collided with "qna failed".)

Inventory

# hosts.toml
[defaults]
qna_version = "11.0"        # version spec; overridable per host

[hosts.mac-test]            # table name is the ~/.ssh/config alias
transport = "ssh"
become = true               # sudo for root-only inspectors

[hosts.this-controller]
transport = "local"          # no `become` line needed: implied on a macOS controller

[hosts.ubuntu-22]
transport = "container"
image = "ubuntu:22.04"

As a library

from bigfix_remote_client_relevance import Target, evaluate_client_relevance

results = await evaluate_client_relevance(
    "name of operating system",
    [Target(kind="container", name="ubuntu:22.04", image="ubuntu:22.04")],
    qna_version="11.0",
)

For results as they arrive rather than all at once — live progress, or an MCP tool streaming partial output — iterate the streaming variant instead:

from bigfix_remote_client_relevance import evaluate_client_relevance_stream

async for result in evaluate_client_relevance_stream("name of operating system", targets):
    print(result.host, result.answers)

It yields in completion order; evaluate_client_relevance waits for the whole fan-out and returns target-then-version order.

One ClientRelevanceResult comes back per (target × version), carrying answers, answer_types, error / error_kind, the resolved qna_version, and the full raw_qna_output for debugging. Failures are reported inside results rather than raised, so one unreachable host never breaks a fan-out.

The library logs through logging and never writes to stdout — that channel belongs to the CLI's payload, and to a stdio MCP server's JSON-RPC.

The package ships py.typed, so mypy and pyright see real types across the import boundary.

Building an MCP server on this

The whole surface an MCP tool needs is here, in stdlib-only helpers — nothing below adds a dependency to your server.

From Python

from bigfix_remote_client_relevance import (
    BigFixRelevanceError,
    Target,
    count_work,
    evaluate_client_relevance_stream,
    format_results,
    result_to_dict,
)

targets = [Target(kind="container", name="ubuntu:22.04", image="ubuntu:22.04")]

total = count_work(targets, "11.0")          # the progress denominator
results = []
async for result in evaluate_client_relevance_stream(expr, targets, qna_version="11.0"):
    results.append(result)
    await report_progress(len(results), total)

return {
    "content": [{"type": "text", "text": format_results(results)}],
    # max_raw_output caps qna's transcript, which is unbounded by default.
    "structuredContent": {"results": [result_to_dict(r, max_raw_output=4000) for r in results]},
    "isError": any(not r.ok for r in results),
}

The fan-out functions never raise for a target failure — an unreachable host comes back as a result with error_kind set — so only the setup path needs the try. Cancelling the fan-out (an MCP request cancellation) propagates CancelledError normally rather than being turned into a result.

Result payloads only ever gain fields within a major version; SCHEMA_VERSION bumps when they do, and RESULT_JSON_SCHEMA does not set additionalProperties: false so an older validator keeps working.

From a non-Python server

Shell out to the CLI, which is the same code paths:

bigfix-remote-client-relevance --schema

prints the JSON Schema for a single result and exits 0 — no target needed. Then --jsonl emits exactly one of those objects per line, flushed as each target answers, so a Node or Go server can stream progress off the pipe. --json emits the whole array once. stdout carries only the payload; logs and error summaries go to stderr, at a level set by -v/-vv.

If several server processes share a machine, they also share the qna artifact cache — safely and efficiently. Downloads stage to a temp name, verify their published sha256, and land by atomic rename, so a race can never serve a partial or corrupt artifact; a filelock-backed cross-process lock means two processes starting at once share one download rather than each fetching it. A stuck download only ever blocks a live wait — the lock is released by the OS the instant a holder crashes, no stale-lock cleanup needed — bounded by ensure_artifact's lock_timeout_s (10 minutes by default).

How qna gets to the target

The machine running the CLI (the controller) owns downloading agent artifacts from support.bigfix.com and pushes them out; targets never fetch from the internet. Ten hosts on the same version cost one download, which also means it works against lab endpoints with no outbound access.

A version spec resolves at run time — 11.0 picks the newest patch in that stream, 11.0.6.137 pins exactly — and the resolved full version is what gets recorded in every result. Artifacts are checksum-verified against the release site's published SHA256SUMS and cached forever, since they are immutable per version. On a target, an extracted version is left in place, so it crosses the wire once ever rather than once per run.

Only Windows has a standalone QnA download; every other platform extracts qna out of the agent package without installing it.

Comparing across targets

Across several targets the useful answer is usually not one block per target, it is where they disagree. --diff collapses identical answers:

bigfix-remote-client-relevance \
  --container ubuntu:22.04 --container debian:12 \
  --container almalinux:9 --container rockylinux:9 \
  --qna-version 11.0 --diff "number of properties"

== group 1 (2 targets)
-- container:ubuntu:22.04@x86_64 (qna 11.0.6.137)
-- container:debian:12@x86_64 (qna 11.0.6.137)
2134

== group 2 (2 targets)
-- container:almalinux:9@x86_64 (qna 11.0.6.137)
-- container:rockylinux:9@x86_64 (qna 11.0.6.137)
2151

When everything agrees it says so once, which is the quickest way to check that a set of platforms answers identically. Answer types count as part of the answer; the qna version does not, so --qna-version 11.0 --qna-version 10.0 --diff tells you whether the two versions agree. --diff is a text summary — for machine consumption use --json on its own.

Containers

With --container, an unspecified platform is probed, not assumed — the image is asked what it is (the same check SSH runs, including over --inventory, and for the same reason: a wrong guess silently hands the box a .deb or an .rpm it can't extract), and an unrecognized answer fails loudly rather than silently running the wrong agent. Pass --platform to skip the probe or override it for a fleet.

The qna artifact is extracted once on the controller and bind-mounted in, so an image needs no package manager of its own — rpm2cpio/cpio or dpkg-deb/ar/tar are never required inside a container. The first run against a given (image, qna version, arch) also builds a small derived image with the tree baked in and reuses it on every later run against the same combination — no mount, no extraction, sub-second start. Pass --rebuild-image to force a fresh one.

While building that image, qna is checked to see that it actually starts. Minimal images often lack a shared library it needs — rockylinux:9 and amazonlinux:2023 have no libdbus-1.so.3 — so the missing package is installed and baked in, paid once rather than per run. --no-auto-setup turns that off for air-gapped hosts; the run then fails naming the library rather than installing anything. A failed or skipped install is never committed, so nothing broken is cached.

These derived images are tagged bfrcr/prepared:* and are safe to remove at any time:

docker rmi $(docker images 'bfrcr/prepared:*' -q)

--arch defaults to this host's own architecture (so on Apple Silicon a bare --container ubuntu:24.04 now targets arm64, not the docs' x86_64 examples above). It is also repeatable, to evaluate more than one architecture in a single run — useful on Apple Silicon, which runs arm64 containers natively and amd64 ones emulated (via Rosetta/QEMU), both at the same time:

bigfix-remote-client-relevance \
  --container ubuntu:24.04 --arch amd64 --arch arm64 "name of operating system"

--engine picks the container engine: auto (default, prefers Docker and falls back to podman only if Docker is unreachable), docker, or podman.

Requirements

• Python 3.11+
• Docker or podman for --container; SSH access for remote hosts
• On macOS, qna needs root — --local implies --become there automatically (pass --no-become to opt out); over SSH it stays opt-in, since the remote platform isn't known up front. --become uses sudo -n, so it needs passwordless sudo or a cached credential; it never prompts.

SSH host keys are verified against ~/.ssh/known_hosts like the ssh CLI, so a brand-new endpoint needs its key trusted first. For throwaway lab hosts, --insecure-skip-host-key-check skips that at the cost of the connection's protection against interception.

Windows endpoints need OpenSSH server and nothing else — commands are invoked through powershell.exe explicitly, so the stock cmd.exe default shell works and no registry change is required.

Development

uv sync
uv run pytest

The unit suite runs offline on a bare machine. Tests needing a real qna binary, Docker, sshd, or the network are marked and auto-skip — see DESIGN.md § Testing.

Prior art

• jgstew/EvaluateRelevance — the local qna wrapper and output parser this ports.
• jgstew/tools — the canonical bash/bigfix_run_qna_*.sh and CMD/bigfix_run_qna_win.bat bootstrap scripts, ported here into bootstrap/ with the pinned version turned into a parameter and the download moved to the controller.
• jgstew/remote_relevance — an earlier action-deployment approach, superseded by the SSH and container transports.

Demo

License

MIT — see LICENSE.