tlidb 0.0.3

The Transfer Learning in Dialogue Baselines Toolkit

The Transfer Learning in Dialogue Benchmarking Toolkit

Overview

---

TLiDB is a tool used to benchmark methods of transfer learning in conversational AI. TLiDB can easily handle domain adaptation, task transfer, multitasking, continual learning, and other transfer learning settings. TLiDB maintains a unified json format for all datasets and tasks, easing the coding process for new tasks. We highly encourage community contributions to the project.

The main features of TLiDB are:

1. Dataset class to easily load a dataset for use across models
2. Unified metrics to standardize evaluation across datasets
3. Extensible Model and Algorithm classes to support fast prototyping

Installation

---

To use TLiDB, you can simply isntall via pip:

pip install tlidb

OR, if you would like to edit or contribute, you can clone the repository and install from source:

git clone [e-mail]:alon-albalak/TLiDB.git
cd TLiDB
pip install -e .

examples/ contains sample scripts for:

1. Training/Evaluating models in transfer learning settings
2. 3 example models: BERT, GPT-2, T5, and training algorithms for each

How to use TLiDB

---

TODO:

• Add examples for using examples/run_experiment.py
• Add examples for data loading/training

Using the example scripts

TLiDB has example scripts to be used for training and evaluating models in transfer learning settings.

Data Loading

TLiDB offers a simple, unified interface for loading datasets. The following example shows how to load the data, and put the data into a dataloader:

from TLiDB.datasets.get_dataset import get_dataset
from TLiDB.data_loaders.data_loaders import get_loader

# load the dataset, and download if necessary
dataset = get_dataset(
    dataset='DailyDialog',
    task='emotion_recognition',
    dataset_folder='TLiDB/data',
    model_type='Encoder', #Options=['Encoder', 'Decoder','EncoderDecoder']
    split='train',#Options=['train', 'dev', 'test']
    )

# get the dataloader
dataloader = get_data_loader(
    split='train', 
    dataset=dataset,
    batch_size=32,
    model_type='Encoder'
    )

# train loop
for batch in dataloader:
    X, y, metadata = batch
    ...

Folder descriptions:

---

• /TLiDB is the main folder holding the code for data
  • /TLiDB/data_loaders contains code for data_loaders
  • /TLiDB/data is the destination folder for downloaded datasets
  • /TLiDB/datasets contains code for datasets
  • /TLiDB/metrics contains code for loss and evaluation metrics
  • /TLiDB/utils contains utility files
• /examples contains sample code for training models
  • /examples/algorithms contains code which trains and evaluates a model
  • /examples/models contains code to define a model
  • /examples/configs contains code for model configurations
  • /examples/logs_and_models is the destination folder for training logs and model checkpoints
• /dataset_preprocessing is for reproducability purposes, not required for end users. It contains scripts used to preprocess the TLiDB datasets from their original form into the TLiDB form