unKR 0.9.2

An Open Source Library for uncertain Knowledge Reasoning

unKR: A Python Library for Uncertain Knowledge Graph Reasoning by Representation Learning

English | 中文

unKR is an python library for uncertain Knowledge graph (UKG) Reasoning based on the PyTorch Lightning. It provides a unifying workflow to implement a variety of uncertain knowledge graph representation learning models to complete UKG reasoning. unKR consists of five modules: 1) Data Processor handles low-level dataset parsing and negative sampling, then generates mini-batches of data; 2) Model Hub implements the model algorithms, containing the scoring function and loss function; 3) Trainer conducts iterative training and validation; 4) Evaluator provides confidence prediction and link prediction tasks to evaluate models' performance; 5) Controller controls the training worklow, allowing for early stopping and model saving. These modules are decoupled and independent, making unKR highly modularized and extensible. Detailed documentation of the unKR is available at here.

unKR core development team will provide long-term technical support, and developers are welcome to discuss the work and initiate questions using issue.

💻 Demo

This is a demo shows the training and testing process of PASSLEAF model with unKR.

Datasets

unKR provides three public UKG datasets including CN15K, NL27K, and PPI5K. The following table shows the source, the number of entities, relations, and facts of each dataset.

Dataset	Source	#Entity	#Relation	#Fact
CN15K	ConceptNet	15000	36	241158
NL27K	NELL	27221	404	175412
PPI5K	STRING	4999	7	271666

📝 Models

Now, nine uncertain knowledge graph representation learning models are available and they can be divided to two types: normal and few-shot models.

Type	Model
Normal	BEURrE, FocusE, GTransE, PASSLEAF, UKGE, UKGsE, UPGAT
Few-shot	GMUC, GMUC+

Reproduce Results

unKR determines two tasks, confidence prediction and link prediction, to evaluate models' ability of UKG reasoning. For confidence prediction task, MSE (Mean Squared Error) and MAE (Mean Absolute Error) are reported. For link prediction task, Hits@k(k=1,3,10), MRR (Mean Reciprocal Rank), MR (Mean Rank) under both raw and filterd settings are reported.

Here are the reproduce results of nine models on NL27K dataset with unKR. See more results at here.

Confidence prediction

Type	Model	MSE	MAE
Normal	BEUrRE	0.08920	0.22194
	PASSLEAF_ComplEx	0.02434	0.05176
	PASSLEAF_DistMult	0.02309	0.05107
	PASSLEAF_RotatE	0.01949	0.06253
	UKGElogi	0.02861	0.05967
	UKGElogiPSL	0.02868	0.05966
	UKGErect	0.03344	0.07052
	UKGErectPSL	0.03326	0.07015
	UKGsE	0.12202	0.27065
	UPGAT	0.02922	0.10107
Few-shot	GMUC	0.01300	0.08200
	GMUC+	0.01300	0.08600

Link prediction (filter on high-confidence test data)

Type	Model	Hits@1	Hits@3	Hits@10	MRR	MR
Normal	BEUrRE	0.156	0.385	0.543	0.299	488.051
	FocusE	0.814	0.918	0.957	0.870	384.471
	GTransE	0.222	0.366	0.493	0.316	1377.564
	PASSLEAF_ComplEx	0.669	0.786	0.876	0.741	138.808
	PASSLEAF_DistMult	0.627	0.754	0.856	0.707	138.781
	PASSLEAF_RotatE	0.687	0.816	0.884	0.762	50.776
	UKGElogi	0.526	0.670	0.805	0.622	153.632
	UKGElogiPSL	0.525	0.673	0.812	0.623	168.029
	UKGErect	0.509	0.662	0.807	0.609	126.011
	UKGErectPSL	0.500	0.647	0.800	0.599	125.233
	UKGsE	0.038	0.073	0.130	0.069	2329.501
	UPGAT	0.618	0.751	0.862	0.701	69.120
Few-shot	GMUC	0.335	0.465	0.592	0.425	58.312
	GMUC+	0.338	0.486	0.636	0.438	45.774

🛠️ Usage

Installation

Step1 Create a virtual environment using Anaconda and enter it.

conda create -n unKR python=3.8
conda activate unKR
pip install -r requirements.txt

Step2 Install package.

• Install from source

git clone https://github.com/seucoin/unKR.git
cd unKR
python setup.py install

• Install by pypi

pip install unKR

Data Format

For normal models, train.tsv, val.tsv, and test.tsv are required.

For few-shot models, train_tasks.json, dev_tasks.json, test_tasks.json and path_graph are required.

UKGE model:
    softloic.tsv: (ent1, rel, ent2, score)
GMUC, GMUC+ models:
    train/dev/test_tasks.json: {rel:[[ent1, rel, ent2, score], ...]}
    path_graph: (ent1, rel, ent2, score)
    ontology.csv: (number, h, rel, t)

train.tsv: All triples used for training and corresponding confidence scores in the format(ent1, rel, ent2, score), one quaternion per line.

val.tsv: All triples used for validation and corresponding confidence scores in the format(ent1, rel, ent2, score), one quaternion per line.

test.tsv: All triples used for testing and corresponding confidence scores in the format(ent1, rel, ent2, score), one quaternion per line.

In UKGE, thesoftlogic.tsvfile is also required.

softlogic.tsv: All triples inferred by PSL rule and corresponding inferred confidence scores in the format(ent1, rel, ent2, score), one quaternion per line.

In GMUC, GMUC+, the following five data files are also needed.

train/dev/test_tasks.json: Few-shot dataset with one task per relation in the format{rel:[[ent1, rel, ent2, score], ...]}. The key of the dictionary is the task name and the values are all the quaternions under the task.

path_graph: All data except training, validation and testing tasks, i.e. background knowledge, in the format(ent1, rel, ent2, score). Each line represents a quaternion.

ontology.csv: Ontology knowledge data required for the GMUC+ model, in the format(number, h, rel, t), one ontology knowledge per line. There are four types of rel, which includes is_A, domain, range, and type.

• c1 is_A c2: c1 is a subclass of c2;
• c1 domain c2: the definition domain of c1 is c2;
• c1 range c2: the value domain of c1 is c2;
• c1 type c2: the type of c1 is c2.

Parameter Adjustment

In the config file, we provide parameter profiles of the reproduced results, and the following parameters can be adjusted for specific use.

parameters:
  confidence_filter:  #whether to perform high-confidence filtering
    values: [0, 0.7]
  emb_dim:
    values: [128, 256, 512...]
  lr:
    values: [1.0e-03, 3.0e-04, 5.0e-06...]
  num_neg:
    values: [1, 10, 20...]
  train_bs:
    values: [64, 128, 256...]

Model Training

python main.py --load_config --config_path <your-config>

Model Testing

python main.py --test_only --checkpoint_dir <your-model-path>

Model Customization

If you want to personalise your own model using unKR, you need to define the following Functions/Classes.

data: Implement data processing functions, including DataPreprocess, Sampler and KGDataModule.

DataPreprocess.py: 
    class unKR.data.DataPreprocess.<your-model-name>BaseSampler
    class unKR.data.DataPreprocess.<your-model-name>Data
Sampler:
    class unKR.data.Sampler.<your-model-name>Sampler
    class unKR.data.Sampler.<your-model-name>TestSampler
KGDataModule.py: 
    class unKR.data.KGDataModule.<your-model-name>DataModule

lit_model: Implement model training, validation, and testing functions.

<your-model-name>LitModel.py:
    class unKR.lit_model.<your-model-name>LitModel.<your-model-name>LitModel

loss: Implement loss functions.

<your-model-name>_Loss.py:
    class unKR.loss.<your-model-name>_Loss.<your-model-name>_Loss

model: Implement model framework functions, classified as UKGModel and FSUKGModel based on whether it is a few-shot model.

<your-model-name>.py:
    class unKR.model.UKGModel/FSUKGModel.<your-model-name>.<your-model-name>

config: Implement parameter settings.

<your-model-name>_<dataset-name>.yaml:
    data_class, litmodel_name, loss_name, model_name, test_sampler_class, train_sampler_class

demo: Implement the model run file.

<your-model-name>demo.py

😊 unKR Core Team

Southeast University: Jingting Wang, Tianxing Wu, Shilin Chen, Yunchang Liu, Shutong Zhu, Wei Li, Jingyi Xu, Guilin Qi.