CoMET-RS — Code Metadata Extraction Toolkit for Research Software

CoMET-RS extracts FAIR (Findable, Accessible, Interoperable, Reusable), machine-actionable metadata from GitHub, GitLab, and Codeberg repositories, and outputs it as JSON-LD conforming to a schema of your choice (ConnOSS, maSMP, or CODEMETA).

It ships as both a command-line tool (comet-rs) and a Python library (comet_rs).

• Source code & full documentation: https://github.com/zbmed-semtec/comet-metadata-extraction
• Issue tracker: https://github.com/zbmed-semtec/comet-metadata-extraction/issues
• License: GPL-3.0

Installation

Requires Python 3.10 or higher.

pip install comet-rs

CLI usage

After installation, the comet-rs command is available on your PATH.

comet-rs --help

usage: comet-rs [-h] {extract,extract_property,fairness} ...

Extract metadata (and per-property sources) from code repositories, for any
supported schema.

Extract full metadata

comet-rs extract <url> <schema> [--schema-class SCHEMA_CLASS] [--token TOKEN] [--with-enrichment]

• url — Repository URL (GitHub, GitLab, or Codeberg).
• schema — Schema to use: connoss, maSMP, or CODEMETA (case-insensitive).
• --schema-class — Schema class to use (default: software).
• --token — Personal access token for the platform. If omitted, CoMET-RS looks for GITHUB_TOKEN, GITLAB_TOKEN, or CODEBERG_TOKEN in the environment, based on the repository URL. A token increases API rate limits and allows access to private repositories.
• --with-enrichment — Include per-property enrichment metadata (source, confidence, category) alongside the extracted JSON-LD.

Example:

comet-rs extract https://github.com/zbmed-semtec/comet-metadata-extraction connoss --with-enrichment

Output (written to stdout as JSON):

{
  "schema": "connoss",
  "code_url": "https://github.com/zbmed-semtec/comet-metadata-extraction",
  "results": { "...": "JSON-LD metadata document..." },
  "enriched_metadata": { "...": "per-property source/confidence/category..." }
}

Extract a single property

comet-rs extract_property <url> <property> [--schema SCHEMA] [--schema-class SCHEMA_CLASS] [--token TOKEN]

• url — Repository URL.
• property — Property name, e.g. name, identifier, codemeta:referencePublication, or codemeta_referencePublication.
• --schema — Schema to use (default: connoss).
• --schema-class — Schema class to use (default: software).
• --token — See above.

Example:

comet-rs extract_property https://github.com/zbmed-semtec/comet-metadata-extraction license

Output:

{
  "property_name": "license",
  "property_value": ["GPL-3.0"],
  "source": "github_api",
  "confidence": 0.95
}

FAIRness assessment

comet-rs fairness <url> [--token TOKEN]

Runs a FAIRness assessment of the repository against the ConnOSS schema.

Example:

comet-rs fairness https://github.com/zbmed-semtec/comet-metadata-extraction

Note: the FAIRness assessment feature is under active revision upstream and may change or be temporarily limited in future releases.

Authentication

CoMET-RS works without authentication, but unauthenticated requests are subject to strict API rate limits on GitHub/GitLab/Codeberg, and cannot access private repositories. Provide a token via --token, or set one of the following environment variables (selected automatically based on the repository URL):

export GITHUB_TOKEN=your_github_token
export GITLAB_TOKEN=your_gitlab_token
export CODEBERG_TOKEN=your_codeberg_token

Python API

CoMET-RS can also be used as a library:

import comet_rs

# Full extraction
result = comet_rs.extract_metadata(
    repo_url="https://github.com/zbmed-semtec/comet-metadata-extraction",
    schema_name="connoss",       # or "maSMP" / "CODEMETA"
    schema_class="Software",
    token=None,                  # or rely on GITHUB_TOKEN / GITLAB_TOKEN / CODEBERG_TOKEN
    with_enrichment=True,
)

# Single property
result = comet_rs.extract_property(
    repo_url="https://github.com/zbmed-semtec/comet-metadata-extraction",
    property_name="author",
    schema_name="connoss",
)
for prop,findings in result.extraction_state.metadata_collector.data.items():
  print(prop)
  for finding in findings:
    print(" -", finding.source, finding.property_value)

Supported schemas

• ConnOSS
• maSMP
• CODEMETA

Supported platforms

• GitHub
• GitLab
• Codeberg

Other ways to run CoMET

CoMET-RS is also available as a web API (via Docker or manual installation) and previously shipped a web frontend (now discontinued/in migration). See the full project documentation on GitHub for Docker Compose instructions, the FastAPI/Swagger backend, and architecture details: https://github.com/zbmed-semtec/comet-metadata-extraction

Citation

If you use CoMET-RS in your research, please cite it using the metadata in CITATION.cff on the GitHub repository: https://github.com/zbmed-semtec/comet-metadata-extraction/blob/main/CITATION.cff

License

CoMET-RS is licensed under the GNU General Public License v3.0 (GPL-3.0). See the LICENSE file for details.