bigfix-relevance-analyzer

A python module for working with BigFix Relevance generically. Extract, Analyze, etc.

This is a library first: it is meant to be depended on by other projects (pre-commit hooks, besapi, MCP servers) rather than run directly.

• No dependencies outside the standard library. (Not "pure Python" - the stdlib XML modules are backed by pyexpat, which is C - but it ships with CPython and PyPy, so there are no wheels to build and no platform matrix.)
• It logs, it never prints. Diagnostics go to the bigfix_relevance_analyzer logger, which gets a NullHandler and nothing else; the library never calls basicConfig or touches your handlers or levels. Nothing is written to stdout, so it is safe to import inside a stdio MCP server, where stray output would corrupt the JSON-RPC stream.

Origin

This project starts from jgstew/pre-commit-bigfix#13, which is the design document for the package: why relevance analysis belongs in a standalone library rather than inside the pre-commit hooks that consume it, what the first milestone covers (a relevance extractor and a heuristic complexity scorer), and the reasoning behind the naming, the dependency choices, and the roadmap. That issue and its comments are the reference for decisions made here; read it before making a structural change.

Roadmap: Python now, possibly Rust later

The short-term goal is pure Python - it keeps iteration fast while the hard part is still unsolved. Relevance has no published grammar, so a real parser means reverse-engineering one from the console, the docs, and real content; that research is the long pole, and Python is the cheapest place to do it.

The parser now exists: a hand-rolled Pratt parser (parser.py) over the existing tokenizer, producing frozen AST nodes (nodes.py) with the operator, precedence, and keyword data kept in declarative tables (grammar.py). The primary asset is the shared corpus of input to expected S-expression parse trees in tests/corpus/*.rlvcorpus - a port is proven equivalent by making the same corpus pass. parse_relevance raises a positioned ParseError; try_parse_relevance never raises, which is the conservative "unknown, skip" interface for scorers and hooks. Every relevance site in the example corpus currently parses; grammar decisions that have not been spot-checked against a real evaluator are tagged [unverified] in their corpus record titles. Not done yet, deliberately: type-directed disambiguation, error-recovery nodes, and rebasing the complexity scorer onto the AST.

The long-term goal may be to translate the core to Rust, exposed as PyO3 wheels for Python consumers and as WebAssembly for a VS Code extension. That is the honest end-state for "one implementation, every consumer": today a Python package can serve pre-commit hooks, besapi, and MCP servers, but it cannot serve an editor. Rust would let the same grammar back both without maintaining two implementations that drift.

Deliberately not started yet: porting during the grammar-research phase would slow the part that is actually hard. Keeping the grammar in declarative tables and the corpus separate from the parser is what makes a later port cheap and provably equivalent - the same corpus has to pass either way. (A tree-sitter grammar was also considered and deferred; it fights relevance's keyword-versus-identifier ambiguity, since relevance has no reserved words and multi-word inspector names.)

Extracting relevance

extract_relevance_from_file finds every relevance statement in a file and reports where each came from and which dialect it is written in:

from bigfix_relevance_analyzer import extract_relevance_from_file

for site in extract_relevance_from_file("MyFixlet.bes"):
    print(f"{site.line}: [{site.dialect.value}] {site.kind} - {site.text}")

Each result is a frozen RelevanceSite with kind, text, line (1-based, in the file), context (a short label for messages), and the dialect fields described under Which dialect a statement is in.

Lower-level entry points (extract_relevance_from_bes_xml, extract_relevance_from_html_text, extract_relevance_from_actionscript, extract_relevance_from_markdown) take content directly, for callers that already have it in hand.

Which dialect a statement is in

Dialect is CLIENT, SESSION, UNCERTAIN or BOTH. Two independent opinions decide it, and every RelevanceSite keeps both rather than collapsing them:

Definite context wins because it is a fact about which engine will evaluate the statement, not an inference. Content fills in the gaps, and a conflict between the two is surfaced rather than resolved away - session inspectors in a fixlet's <Relevance> is relevance in the wrong place, and it fails on every endpoint that evaluates it. Conflicts are logged at WARNING.

The classifier only ever uses positive evidence: an inspector it does not recognize contributes nothing. New BigFix versions add inspectors to both dialects, so an unfamiliar name is never grounds for typing a statement by elimination or for calling it invalid.

One context case is worth knowing about: relevance in HTML or JavaScript is almost always session relevance, but ClientUI dashboards are HTML rendered by the BES Client on the endpoint and hold client relevance, using the identical <?Relevance ?> syntax. What separates them is the mechanism - a ClientUI cannot evaluate relevance from JavaScript at all. So a static substitution in a .html file is read as client relevance, a JavaScript relevance call is always session relevance, and in a file doing both the mechanism settles nothing for its substitutions, leaving their dialect to the content classifier.

Optional lxml adapter

Extraction uses stdlib expat by default. Projects that already parse BES XML with lxml can hand over their existing tree instead of having it parsed twice:

pip install 'bigfix-relevance-analyzer[lxml]'

from bigfix_relevance_analyzer.extract import extract_relevance_from_lxml_tree

sites = extract_relevance_from_lxml_tree(my_tree)

Both paths report identical line numbers, including for a start tag whose attributes span several lines - a test pins this across the whole example corpus, since an off-by-one there would shift every reported line in a file.

Scoring complexity

analyze_relevance_complexity gives a statement a heuristic score, along with the individual metrics that produced it, so a pre-commit hook can threshold on the number and still say why something was flagged:

from bigfix_relevance_analyzer import analyze_relevance_complexity

result = analyze_relevance_complexity(
    'exists files whose (name of it starts with "bes") of folder "/tmp"'
)
print(result.score, result.whose_clauses, result.max_of_chain)

The score covers two different axes. Readability is the token-shaped part: length, nesting, of chains, whose filters. Evaluation cost is what the statement does to the client's eval loop, which does not follow from size - exists descendants of folder "C:\" is eight tokens and walks an entire disk on every evaluation cycle. costly_inspectors names the heavy families that were charged for, so a warning can point at them:

result = analyze_relevance_complexity('exists descendants of folder "C:\\"')
print(result.evaluation_cost, result.costly_inspectors)
# 12.0 ('folder recursion',)

Those families are deliberately not weighted equally - hashing a file is a different order of expense from reading a few lines out of one - and neither is the same family across dialects, when the underlying inspector isn't either.

Cost is also dialect-scoped, per rule rather than per table, and applying to both dialects does not mean costing the same in both. Session relevance cannot read a file at all, so sha1 of <string> is real work but nowhere near sha1 of <file> on a client - the hashing rule charges each accordingly. wmi exists only on a Windows client and results of <bes fixlet> only on the server, so neither is charged against the other dialect at all. Pass the dialect - the extractor already knows it for every site - to get this scoping:

for site in extract_relevance_from_file("MyFixlet.bes"):
    result = analyze_relevance_complexity(site.text, site.dialect)

Without a dialect, nothing is excluded. The client-side families come from the candidate list in jgstew/besapi's examples/fixlet_add_mime_field.py; every inspector name a rule matches on is checked against the QnA dumps by a test, and so is each rule's declared dialect, so the table stays grounded in what BigFix actually defines. Two things are not grounded that way and say so: the tiers are a judgement call rather than a benchmark, and the session-only rules are a seed rather than a survey - there is no curated equivalent of the besapi list for the server side yet. WEIGHT_EVALUATION_COST turns the whole axis off if a consumer only cares about readability.

Counting runs over the token stream, never over raw text, so a comment mentioning whose or the word and inside a string literal cannot inflate the score. The metrics are heuristics and the weights are deliberately module-level constants (WEIGHT_WHOSE_CLAUSE and friends) so they can be tuned against real content without touching the counting.

The tokenizer

bigfix_relevance_analyzer.tokenizer is the lexer the scorer counts against, and the front end the future parser will sit on. It turns text into a lossless stream of tokens: joining their texts reproduces the input exactly, whitespace and comments included, which is what a formatter or auto-fixer would need later. It never raises - malformed relevance yields error tokens, because content extracted from the wild is regularly truncated or broken and a scorer still has to produce a number for it.

It deliberately does not bind multi-word inspector names; that needs the inspector table below and type-directed disambiguation, both of which are parser work. Keeping this layer table-free makes it total: any input lexes, and the same input always lexes the same way, regardless of which dumps happen to exist.

The inspector table

bigfix_relevance_analyzer.inspectors is the structured table of what relevance actually defines - properties, casts, binary and unary operators, and the type universe - parsed from the dumps in tests/examples/relevance_inspectors/.

This is a parser prerequisite, not a parser dependent. Relevance has no reserved words and multi-word inspector names, so nothing about the text of logged on users of bes computers says where one name ends and the next begins; resolving that needs a name table, which is what this is.

from bigfix_relevance_analyzer import inspectors

for entry in inspectors.lookup("drives"):
    print(entry.signature, "->", entry.return_type, sorted(entry.platforms))
# drives -> drive ['windows']
# drives -> filesystem ['debian', 'rhel', 'ubuntu']
# drives -> volume ['macos']

Each row keeps the sources that defined it, so dialects and platforms are derived rather than baked in. That is what makes the example above possible: drives genuinely returns a different type per platform family, and collapsing rows into one "client" verdict would have destroyed that. It is imported explicitly rather than from the package root, since most callers only extract.

The table is a snapshot, not a specification. New BigFix versions add inspectors, and the dumps only cover what someone captured - so absence is grounds for a warning at most, never proof that a name is invalid. Only positive evidence should be drawn from it, the same discipline the dialect classifier applies.

src/bigfix_relevance_analyzer/_inspector_data.py is generated; the dumps are the source of truth. Regenerate after adding or editing one:

python tools/generate_inspector_data.py

A pre-commit hook and tests/test_inspector_data.py both fail if the two have drifted. Dump filenames carry their own provenance as {dialect}_relevance_{category}[_{context}].txt, so a newly captured dump is picked up with no code change.

Development

This project uses uv for dependency management and packaging (build backend: hatchling), with a src/ layout.

uv sync                    # create .venv and install project + dev dependencies
uv run pytest              # run tests
uv run ruff check .        # lint
uv run ruff format .       # format
uv run mypy                # type-check

Set up the git hooks once (the extra hook types let uv-sync re-create .venv after a pull or branch switch, and let the pre-push checks below actually run):

uv run pre-commit install --hook-type pre-commit --hook-type pre-push --hook-type post-checkout --hook-type post-merge

A few slower checks (pytest, uv lock --check, uv build --wheel) are deferred to git push rather than every commit, via stages: [pre-push, manual]. Run them by hand with:

uv run pre-commit run --all-files --hook-stage pre-push

The rest of the manual-only hooks (release/build checks, uv audit, pyproject and GitHub Actions schema validation) don't run automatically at all - CI invokes them with --hook-stage manual, which also picks up the pre-push ones above:

uv run pre-commit run --all-files --hook-stage manual

Dependency freshness delay

pyproject.toml sets [tool.uv] exclude-newer = "7 days", so uv lock/uv sync/uv add only consider package versions that were published at least 7 days ago. This is a rolling window (not a fixed date), giving newly published releases a week to be pulled before this project can depend on them. To deliberately bypass this - for example to pull in an urgent security fix - run:

uv lock --exclude-newer=false