ebes 0.0.5

EBES: Easy Benchmarking for Event Sequences.

EBES Easy Benchmarking for Event Sequences.

EBES is an easy-to-use development and application toolkit for Event Sequence(EvS) Assesment, with key features in configurability, compatibility and reproducibility. We hope this project could benefit both researchers and practitioners with the goal of easily customized development and open benchmarking in EvS.

Setup

Installation

To install the latest stable version:

pip install ebes

Datasets

Dataset	Source Link	Preprocessing Script Link	Download Instructions
Physionet2012	Physionet2012	physionet2012.py	Straightforward download on site
MIMIC-III	MIMIC-III	mimic-3.py	Only credentialed users who sign the DUA can access the files.
Age	Age	age.py	Download here if you have difficulties navigating site
Retail	Retail	x5-retail.py	Download here if you have difficulties navigating site
MBD	MBD	mbd.py	Straightforward download on site
Taobao	Taobao	taobao.py	Need to login on site to download. After that pass tianchi_mobile_recommend_train_user.csv into script
BPI17	BPI17	bpi_17.py	Straightforward download on site
ArabicDigits	ArabicDigits	SpokenArabicDigits.py	Either just run preprocessing script and it will download automatically, or straightforward download on site
ElectricDevices	ElectricDevices	electric_devices.py	Straightforward download on site
Pendulum	We created it ourselves	pendulum.py	Run preprocessing script in order to generate from scratch. Make sure to keep default seed=0 in order to get exactly same dataset.

Usage

python main -d age -m gru -e correlation -s best

Results:

Performance of various models as a function of number of sequences. Metrics from Table 1 are reported. Number of sequences is presented in log scale. Standard deviation across 3 runs is depicted as vertical lines.

Performance metric relationships and correlations of different subsets among all methods on PhysioNet2012 are presented. We do not observe a correlation between the test metric and train-val on PhysioNet2012, as seen in the right upper corner. For the Taobao dataset, we do not observe a clear linear trend between hpo-val and the test metric suggesting the presence of distribution shift.