Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Machine Learning Tool Box

Project description

Machine Learning Tool Box

This is the machine learning tool box. A collection of userful machine learning tools intended for reuse and extension. The toolbox contains the following modules:

  • hyperopt - Hyperopt tool to save and restart evaluations
  • keras - Keras callback for various metrics and various other Keras tools
  • lightgbm - metric tool functions for LightGBM
  • metrics - several metric implementations
  • plot - plot and visualisation tools
  • tools - various (i.a. statistical) tools

Module: hyperopt

This module contains a tool function to save and restart Hyperopt evaluations. This is done by saving and loading the hyperopt.Trials objects. The usage looks like this:

from mltb.hyperopt import fmin
from hyperopt import tpe, hp, STATUS_OK

def objective(x):
    return {
        'loss': x ** 2,
        'status': STATUS_OK,
        'other_stuff': {'type': None, 'value': [0, 1, 2]},

best, trials = fmin(objective,
    space=hp.uniform('x', -10, 10),

print('best:', best)
print('number of trials:', len(trials.trials))

Output of first run:

No trials file "trials_file" found. Created new trials object.
100%|██████████| 100/100 [00:00<00:00, 338.61it/s, best loss: 0.0007185087453453681]
best: {'x': 0.026805013436769026}
number of trials: 100

Output of second run:

100 evals loaded from trials file "trials_file".
100%|██████████| 100/100 [00:00<00:00, 219.65it/s, best loss: 0.00012259809712488858]
best: {'x': 0.011072402500130158}
number of trials: 200

Module: keras

This module provides ROC-AUC- and F1-metrics (which are not included in Keras) in form of a callback. Because the callback adds these values to the internal logs dictionary it is possible to use the EarlyStopping callback to do early stopping on these metrics. The usage looks like this:

bcm_callback = mltb.keras.BinaryClassifierMetricsCallback(val_data, val_labels)
es_callback = callbacks.EarlyStopping(monitor='roc_auc', patience=5,  mode='max')

history =, train_labels, 

                      #do not give validation_data here or validation will be done twice
                      #validation_data=(val_data, val_labels),

                      #always provide BinaryClassifierMetricsCallback before the EarlyStopping callback
                      callbacks=[bcm_callback, es_callback],

Module: lightgbm

This module implements metric functions that are not included in LightGBM. At the moment this is the F1- and accuracy-score for binary and multi class problems. The usage looks like this:

bst = lgb.train(param, 
                feval=mltb.lightgbm.multi_class_f1_score_factory(num_classes, 'macro'),

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for mltb, version 0.1
Filename, size File type Python version Upload date Hashes
Filename, size mltb-0.1-py3-none-any.whl (11.0 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size mltb-0.1.tar.gz (8.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page