CWL Baseline Transpiler Mate Plugin

A standalone plugin implementing the public transpiler-mate-api contract. It compares every Process in two resolved CWL documents and produces an explainable JSON baseline report with a minimum SemVer increment.

Install

python -m pip install cwl-baseline-plugin

For development, install your checkout of the API first if it has not yet been published, then install this project:

python -m pip install -e ../transpiler-mate-api
python -m pip install -e .
hatch run test:test

Requires Python >=3.10, transpiler-mate-api==1.0.0, semver>=3.0.4,<4, and loguru==0.7.3. The API supplies cwl-utils and Pydantic. The effective Python requirement also depends on the selected versions of those dependencies.

Integrate with a transpiler-mate runtime

Exported plugin object: cwl_baseline.baseline_plugin. Plugin name: baseline. Options model: BaselineOptions.

The package registers baseline = "cwl_baseline.plugin:baseline_plugin" in the transpiler_mate.plugins entry-point group. Install it into the same environment as the separately installed runtime. It provides no standalone cwl_baseline executable. The runtime resolves the current CWL; the plugin resolves only the previous release.

transpiler-mate baseline examples/current.cwl \
  --previous examples/previous.cwl --output baseline.json --check

The host CLI resolves its input CWL into context and invokes:

from pathlib import Path
from transpiler_mate.api import TranspilerContext
from cwl_baseline import BaselineOptions, baseline_plugin


def execute_baseline(context: TranspilerContext) -> None:
    baseline_plugin.execute(
        context,
        BaselineOptions(
            previous="path-or-URI-to-previous-release.cwl",
            output=Path("baseline.json"),
            check=True,
        ),
    )

Equivalent options payload for a runtime accepting JSON:

{
  "previous": "path-or-URI-to-previous-release.cwl",
  "output": "baseline.json",
  "check": true
}

previous_context = context.resolver.resolve(options.previous) supplies the baseline. context.document supplies all current Processes, irrespective of context.process_id. Both versions are read from context.metadata.software_version on their respective contexts.

The highest applicable bump is applied once to the previous release using semver.Version.bump_major(), .bump_minor(), or .bump_patch(). The third component is called patch. Neither CWL document nor either metadata object is mutated.

Compatibility policy

The current implementation uses fixed static rules plus the global review_bump classification. It does not yet support configurable per-rule policies. These are plugin baselining assumptions, not versioning rules mandated by CWL.

Replacement requires AcceptedInputs(old) ⊆ AcceptedInputs(new) and PossibleOutputs(new) ⊆ PossibleOutputs(old). Inputs are contravariant; outputs are covariant. Thus string → string | null is minor for an input but major for an output. Identical signatures do not prove compatible behavior.

See the full compatibility policy for the implemented rules and their limitations.

Every Process exposed in context.document is treated as public. A Process addition is minor; a removal or rename is major. This policy is intentionally conservative for documents containing helper tools in their public mapping.

Type comparison is directional: old inputs must be assignable to new inputs; new outputs must be assignable to old outputs. It covers primitives, CWL numeric promotions, nullability, unions, enums, arrays, records, and named schemas. Numeric promotion follows CWL type assignability; it does not claim lossless numeric conversion. Any excludes null unless explicitly unioned with null. Removed record fields are conservatively major for either direction.

Input omission is tracked separately: a serialized default or a nullable type allows omission. Removing a default is major only when omission becomes invalid, with a separate review finding for the default change. Adding or changing an existing parameter's default also requires review.

Hints do not trigger the new mandatory requirement rule. Existing requirement changes, including resource/network settings and container images, generally receive residual behavioral review. Adding a new mandatory container requirement is major. SchemaDefRequirement definitions are compared through parameter types; unused schema changes require review.

File constraints include format and secondaryFiles. Literal format subset relations can prove compatibility, but relationships between different format IRIs may require ontology knowledge and are left for review. Secondary-file expressions are also left for review. Existing optional secondary-file outputs are not treated as required guarantees. Requiring a new secondary input file or weakening a required secondary output guarantee is major; other literal changes are minor. Compatible output-format narrowing has a patch floor without an independent review flag in the current implementation.

Workflow steps are internal implementation details. They are matched by ID; adding/removing one requires behavioral review rather than automatically changing the public version. All residual changed DOM fields are reported, including embedded run objects and unknown extension fields. Lists with execution-significant order (arguments, source lists, etc.) retain their order.

Interpreting the report

{
  "schema_version": "1.0",
  "previous_version": "1.2.3",
  "current_version": "1.2.3",
  "minimum_bump": "major",
  "minimum_version": "2.0.0",
  "suggested_version": "2.0.0",
  "review_required": false,
  "review_bump": null,
  "declared_version_sufficient": false,
  "findings": []
}

The abbreviated example omits findings. Real findings contain a stable rule identifier, JSON-Pointer-style comparison path, category, minimum bump, before/after values, presence flags, explanation, and review flag. Paths refer to the normalized comparison model, not source-file line numbers.

• minimum_bump and minimum_version express the statically established floor.
• suggested_version is null while review remains unresolved. Otherwise it is the minimum proposed release, not an instruction to downgrade a higher already-declared version.
• A report without changes retains the previous release version.
• declared_version_sufficient is false while review is unresolved, even if the current version is numerically high enough.
• The full report is written before check=True raises PluginFailureError.
• Missing/invalid SemVer metadata and prerelease baselines are domain errors. Current development prereleases are accepted as input, but 1.3.0-dev.1 does not satisfy a required final 1.3.0 release. Build metadata is handled by the semver library.

After reviewing behavioral changes, classify them explicitly:

options = BaselineOptions(
    previous="release.cwl",
    output="baseline.json",
    review_bump="patch",  # All review findings have been assessed as compatible fixes.
    check=True,
)
baseline_plugin.execute(context, options)

review_bump applies to all review findings in this invocation. It cannot reduce a proven major/minor requirement. Original per-finding review flags are retained for auditing, while the report-level flag indicates unresolved review. A global classification is not a per-rule waiver mechanism.

For library use without writing files:

from cwl_baseline import baseline

previous = context.resolver.resolve("release.cwl")
report = baseline(previous, context)
print(report.model_dump_json(indent=2))

Normalization and boundaries

The plugin serializes the existing cwl-utils DOMs using save(relative_uris=False). It strips only the owning document's location from CWL identity/reference fields, retaining scopes and external URIs. It does not rewrite literal defaults, command arguments, expressions, or extension values. Unambiguous short named-type references are recognized within a Process; ambiguous/unresolved references produce review when their compatibility cannot be established. Reachable named schemas are compared through their parameters; changes to otherwise unused declarations are reported for review.

The runtime owns CWL parsing, import resolution, graph validation, and metadata extraction. This plugin does not execute CWL, evaluate JavaScript, inspect image contents, fetch format ontologies, or recursively fetch additional run URLs. For reproducible baselines, the resolver must provide pinned dependency snapshots in document. Changes behind an unchanged external reference that is absent from the supplied DOMs cannot be detected. Inherited or externally defined schemas not present in a Process's available declarations may require review.

Generated serializers may discard source spelling, comments, and explicit null defaults. The comparison concerns the effective supplied DOM representation; it cannot recover syntax that the parser discarded. Anonymous/generated public Process IDs should be replaced with stable public IDs before baselining. Generated IDs of anonymous inline tools are ignored; their enclosing step/run path identifies them.

Static type compatibility does not prove scientific or behavioral equivalence. A reviewed major release can still be necessary when output meanings or units change without changing CWL types.

Tests and examples

tests/ exercises parsed CWL DOMs, plugin resolution and error semantics, variance, schema references, record/array changes, file constraints, source relocation, ordered arguments, steps, and version aggregation. Test fixtures use synthetic source URIs and disable link-existence checking only in their fixture loader; production loading remains entirely with the host resolver.

examples/previous.cwl, examples/current.cwl, and examples/baseline.json illustrate adding an optional input. Run hatch run test:test to execute the configured Python-version test matrix.

Upstream API: https://github.com/transpiler-mate/transpiler-mate-api SemVer library: https://pypi.org/project/semver/

Documentation

Project documentation is published at: https://Transpiler-Mate.github.io/cwl-baseline-plugin/

Contribute

Submit a Github issue if you have comments or suggestions.

Local quality checks

Install Hatch and Taskfiles then install the Git hook:

task quality:pre-commit:install

Every commit runs Ruff (including the configured McCabe complexity limit), Ruff formatting, mypy, Bandit, and the pytest suite.

Run the complete hook explicitly with:

task quality:pre-commit:run

License