SKOS vocabulary management with GitHub & Excel

Overview

For voc4cat, a term collection for catalysis created in NFDI4Cat, we developed a toolbox for collaboratively maintaining SKOS vocabularies on GitHub using Excel (xlsx-files) as user-friendly interface. It consists of several parts:

• voc4cat-tool (this package)
  • A command-line tool to convert vocabularies from Excel/xlsx to SKOS (turtle/rdf) and validate the vocabulary. Validation includes formal SHACL profile checks as well as additional custom validation. The voc4cat tool can be run locally but is also well suited for integration in CI-pipelines. It was inspired by RDFLib/VocExcel. Parts of the VocExcel codebase were merged into this repository (see git history).
• voc4cat-template
  • A GitHub project template for managing SKOS-vocabularies using a GitHub-based workflows including automation by gh-actions.
• voc4cat
  • A SKOS vocabulary for the catalysis disciplines that uses the voc4cat workflow for real work.

Command-line tool voc4cat

voc4cat was mainly developed to be used in gh-actions but it is also useful as a locally installed command line tool. It has the following features.

• Convert between SKOS-vocabularies in Excel/xlsx format and rdf-format (turtle) in both directions.
• Check/validate SKOS-vocabularies in rdf/turtle format with the vp4cat SHACL-profile, a derivative of vocpub.
• Manage vocabulary metadata (title, description, creator, publisher, etc.) via configuration file.
• Extract provenance information from git history (created, updated).
• Allocate ID ranges to contributors and track their contributions.
• Check xlsx vocabulary files for errors or incorrect use of IDs (voc4cat uses pydantic for this validation).
• Generate documentation from SKOS/turtle vocabulary file using pyLODE.

The RDF produced by 1.x version of voc4cat-tool is well aligned with the data model of Skosmos 3 (even deprecations are correctly visualized). We recommend Skosmos for providing an API to your vocabulary or if you want as a richer user interface than the static HTML documentation.

Installation

voc4cat is platform independent and works on Windows, Linux, and macOS. It requires Python (3.10 or newer).

If you only want to use the command-line interface, it is strongly suggested to install with uv tool or pipx. Both simplify installing and managing Python command-line applications.

uv tool install voc4cat

or

pipx install voc4cat

To validate the successful installation, run

voc4cat --version

The available commands and options can be explored via the help system:

voc4cat --help

The voc-assistant command comes with the default install. voc-assistant check VOCAB screens one vocabulary for similar concepts; voc-assistant compare PUBLISHED SUBMITTED screens only what the submission adds. Both read idranges.toml, where accepted_similarity records the concept pairs you have reviewed and concept_url_template sets where the report links to. Run voc-assistant check --help for the thresholds and the other options.

Out of the box it compares text with the Levenshtein ratio, which needs no model:

voc-assistant check myvocab.ttl --method levenshtein --definitions none

That reports the concept pairs whose labels reach --threshold-labels-certain and counts the ones it could not judge. Since Levenshtein scores spelling variants below the 0.98 default (co-precipitation against coprecipitation is 0.9677), lower the threshold to catch them.

For semantic similarity, which also compares definitions and is what the default --method sbert uses, install the sbert extra. It pulls in PyTorch and takes roughly 800 MB of disk space, and the embedding model it downloads on first use takes another 275 MB, so it is not part of the default install:

uv tool install "voc4cat[sbert]"

In a clone of this repository, uv sync --extra sbert adds it to the development environment.

Alternatively, you can install voc4cat using pip like any other Python package.

For development, uv sync installs voc4cat together with all development tools. With pip the same set is available as dependency groups: pip install -e . --group dev (requires pip >= 25.1).

Getting started

See the Documentation for detailed guidance.

To create a new vocabulary, first set up a configuration file idranges.toml for your vocabulary (see example). This file defines vocabulary metadata and ID ranges for contributors. Then create an xlsx-template:

voc4cat template --config myvocab/idranges.toml --outdir myvocab/ myvocab

This creates myvocab.xlsx (named after your vocabulary) with the structure for entering concepts.

Convert the vocabulary file from xlsx to SKOS/turtle format:

voc4cat convert --config myvocab/idranges.toml myvocab/myvocab.xlsx

A turtle file myvocab.ttl is created in the same directory.

The reverse is also possible. Create an xlsx file from a turtle vocabulary:

voc4cat convert --config myvocab/idranges.toml --outdir myvocab/ myvocab/myvocab.ttl

Migrating from older versions

Vocabularies created with voc4cat-tool v0.10.x or earlier (format "043") can be converted to the v1.0 format. See migrating to v1.0 for details.

Feedback and code contributions

We highly appreciate your feedback. Please create an issue on GitHub.

Before you contribute code, we suggest to first create an issue to get early feedback on your ideas before you spend too much time.

By contributing, you agree that your contributions are licensed under the project's BSD-3-Clause license.

Contributors

For details, see the Zenodo record.

A big thanks to our GitHub contributors:

Figure made with contrib.rocks.

Acknowledgement

This work was funded by the German Research Foundation (DFG) through the project "NFDI4Cat - NFDI for Catalysis-Related Sciences" (DFG project no. 441926934), within the National Research Data Infrastructure (NFDI) programme of the Joint Science Conference (GWK).

This project includes the vocpub SHACL profile, which is licensed under the Creative Commons Attribution 4.0 International License (CC-BY 4.0) and was created by Nicholas J. Car. A copy of the license can be found at: https://creativecommons.org/licenses/by/4.0/.