semantic-benchmark

Reusable Python helpers for semantic benchmark descriptions, aggregate RO-Crate creation, and RoHub provenance workflows.

Install

pip install semantic-benchmark

Install optional features with extras:

pip install "semantic-benchmark[rocrate]"
pip install "semantic-benchmark[all]"

To install the latest unreleased version directly from GitHub:

pip install "semantic-benchmark[all] @ git+https://github.com/Simulation-Benchmarks/semantic-benchmark.git"

Provided Modules

• semantic_benchmark.semantics: dataclasses and BenchmarkLoader for JSON-LD benchmark descriptions.
• semantic_benchmark.rohub: RoHub configuration, upload, download, annotation, and query helpers.
• semantic_benchmark.rocrate: aggregate RO-Crate creation and validation helpers.
• semantic_benchmark.runner: shared parameter-file, workspace, resource staging, archive, logging, and aggregate RO-Crate helpers for project benchmark runners.

semantic_benchmark.semantics is available from the base installation. semantic_benchmark.rocrate requires the rocrate extra. semantic_benchmark.rohub is available from the base installation.

The root semantic_benchmark package re-exports the semantic classes for backwards compatibility. The legacy semantic_benchmark.semantic module also remains as a compatibility shim.

Loaded numerical variables and parameters expose both unit (the original JSON-LD value) and unit_iri (the full IRI expanded through the document's namespace bindings). Consumers should use unit_iri when creating links.

semantic_benchmark.rohub.download_benchmark_resources(...) downloads the requested benchmark resources from a RoHub research object. The semantic file is selected from the Annotation Collection resource in list_resources(). Passing path="benchmark" (CLI: --path benchmark) exports the complete research object using ros_export_to_rocrate(..., use_format="zip") and extracts its contents directly into benchmark/. If path / --path is omitted, the crate is extracted into the current directory. The directory is created if needed, and the temporary ZIP is removed afterward. The returned mapping contains the research object identifier under RO-Crate and the annotation resource identifier under Annotation Collection for the requested downloads. The package also exposes the download-semantic-benchmark CLI.

upload-semantic-benchmark calls upload_provenance_rocrate(...) directly. It accepts --provenance_folderpath (the RO-Crate ZIP), --benchmark-name, --username, --password, and optional --code-repository-url, --used-software-url, and --use-production-rohub arguments.

semantic_benchmark.rocrate.create_main_ro(...) can validate the generated aggregate crate by passing validation_profile. The package writes the RO-Crate zip, unpacks it to a validation directory, and runs validate_rocrate(...).

Repository-specific projects can override the packaged RoHub defaults with:

import semantic_benchmark.rohub as rohub

rohub.configure_repository_settings(
    rohub_config={...},
)