OnT (Language Models as Ontology Encoder)

Project Overview

OnT is a language model-based framework for ontology embeddings, enabling effective representation of concepts as points in hyperbolic space and axioms as hierarchical relationships between concepts. Built upon the HierarchyTransformer, this implementation provides enhanced capabilities for ontological reasoning through specialized embedding techniques such as concept rotation, transition, and existential quantifier representation. The model has been trained on various biomedical ontologies including GO, GALEN, and ANATOMY datasets. For the test of geometric-embedding methods, please refer to the BoxSquaredEL and TransBox.

Features

• Hyperbolic embeddings for ontology concept encoding
• Modeling of hierarchical relationships between concepts
• Support for role embeddings as rotations over hyperbolic spaces

Installation via pip

The easiest way to get started is to install the package from PyPI ontology-transformer:

pip install ontology-transformer

The distribution is named ontology-transformer, while its Python import package is lowercase ont. The root-level OnT.py file is retained as legacy research code and is not the module installed from PyPI.

You can fine-tune the model directly from your OWL ontology file without any manual preprocessing:

from ont import OntologyTransformer
 
# Simply provide your OWL file path - that's it!
model = OntologyTransformer.fit(
    owl_path="path/to/your/ontology.owl",  # Your ontology file
    output_dir="./my_ontology_model",
    num_epochs=5,
    batch_size=128,
    eval_ratio=0.1  # Use 10% for evaluation
)
 
# The model is now fine-tuned on your ontology
# Use it immediately for encoding
embeddings = model.encode(["YourConcept1", "YourConcept2"])
 
# Save for later use
model.save("./my_ontology_model/final")

Project Structure

The data and models folders should be downloaded and unzipped to the root directory. The Google Drive links are all anonymous.

• OnT.py: Main model implementation containing the OntologyTransformer class

• data/: Contains training and testing data, download from here or from Zenodo View Dataset on Zenodo

• models/: Stores pre-trained and fine-tuned models, download from here or from huggingface View Models on Hugging Face

• normalization/: Scripts for normalizing the EL part of a given ontology. See normalization/Readme.md for details.

Installation from Github

We recommend using Conda for a reproducible setup. The steps below install PyTorch, HierarchyTransformers, and a compatible sentence-transformers version, then perform a quick verification.

1) Create and activate Conda environment

conda create -y -n ont python=3.12
conda activate ont

2) Install PyTorch (GPU, CUDA 12.1)

conda install -y pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

3) Install HierarchyTransformers and sentence-transformers

• ontology-transformer supports sentence-transformers>=3.0, including 6.x; no 3.4.0 pin is required.

pip install git+https://github.com/KRR-Oxford/HierarchyTransformers.git
pip install 'sentence-transformers>=3.0'

Usage

Then load the model and use it for inference or training as follows.

import torch
from ont import OntologyTransformer

# Load from Hugging Face (recommended)
ont = OntologyTransformer.from_pretrained('Hui97/OnT-MiniLM-L12-galen')


# entity names to be encoded.
entity_names = ["continuant", "occurrent", "independent continuant", "process"]

# get the entity embeddings
entity_embeddings = ont.encode(entity_names)

# role sentences to be encoded.
role_sentences = ["application attribute", "attribute", "chemical modifier", "chemical process modifier attribute"]

# get the role embeddings, consist of the rotation and scaling (regarded as 1 by default for model uploaded to huggingface)
role_rotations, _ = ont.encode_roles(role_sentences)

For training, run the following command. Remember to update the dataset_path and dataset_name in config.yaml :

python train_ont.py -c config.yaml

Available Pre-trained Models

The following models are available on Hugging Face and can be loaded directly:

All models can be loaded using:

from ont import OntologyTransformer
ont = OntologyTransformer.from_pretrained('Hui97/OnT-MiniLM-L12-galen')