Skip to main content

A tool to evaluate the performance of various machine learning algorithms and preprocessing steps to find a good baseline for a given task.

Project description

A tool to evaluate the performance of various machine learning algorithms and preprocessing steps to find a good baseline for a given task.

Installation

pip install evaluate

Example

import evaluate
from sklearn import datasets

data = datasets.load_iris()
x, y = data.data, data.target

results = evaluate(task='classification', data=(x, y))
results['test_score'].plot.bar()

Documentation

This tool performs common preprocessing steps such as feature scaling, one-hot encoding etc., and runs various ML algorithms such as Random Forests, SVM etc. It then evaluates the performance of each preprocessing step and ML algorithm and provides scores for each. These results can be used to quickly identify preprocessing steps and ML algorithms that perform well to form a good baseline which can be used to develop better models.

evaluate(task,
         data,
         test_data=.2,
         columns=None,
         preprocessors=None,
         estimators=None)
Args
  • task: 'classification' or 'regression'
  • data: Tuple of x, y used for training the model
  • test_data: Tuple of x, y or a number representing the proportion of data to be used for scoring the model
  • columns: Dictionary of lists mapping column types to column names. If not specified numeric and categorical columns are automatically identified
  • preprocessors: List of names of available preprocessors or a custom Preprocessors object
  • estimators: List of names of available estimators or a custom Estimators object
Returns

Dictionary of pandas DataFrames with estimator names as index and preprocessor names as column names with the following keys:

{
    'test_score': ...,
    'train_score': ...,
    'fit_time': ...,
    'score_time': ...,
}
results = evaluate(...)
assert isinstance(results, dict)
scores = results['test_score']
assert isinstance(scores, pandas.DataFrame)
scores.plot.bar()

Preprocessors

Available Preprocessors

Name Column Type Description
n numeric Handle missing data
n:s numeric Standardize features
c categorical Handle missing data and perform one-hot encoding
o ordinal Handle missing data and perform ordinal encoding
t:c text Convert to a matrix of token counts
t:c=2 text Convert to a matrix of token counts including bigrams
t:t text Convert to a matrix of TF-IDF features
t:t=2 text Convert to a matrix of TF-IDF features including bigrams

Multiple preprocessors can be combined into one by separating them with ,:

results = evaluate(..., preprocessors=['n,c,o', 'n:s,c,o'])

Custom Preprocessors

Custom preprocessors can be added as:

from evaluate import evaluate, Preprocessors

preprocessors = Preprocessors()
preprocessors.add('custom_preprocessor', CustomPreprocessor())
results = evaluate(..., preprocessors=preprocessors)

Name of the custom preprocessor must be unique.

Estimators

Available Estimators

Classification Regression
XGBClassifier XGBRegressor
LGBMClassifier LGBMRegressor
RandomForestClassifier RandomForestRegressor
SVC SVR
LogisticRegression LinearRegression
KNeighborsClassifier KNeighborsRegressor
AdaBoostClassifier AdaBoostRegressor
ExtraTreesClassifier ExtraTreesRegressor
GradientBoostingClassifier GradientBoostingRegressor
DecisionTreeClassifier DecisionTreeRegressor
DummyClassifier DummyRegressor

Custom Estimators

Custom estimators can be added as:

from evaluate import evaluate, Estimators

estimators = Estimators(task='classification')
estimators.add('custom_estimator', CustomEstimator())
results = evaluate(..., estimators=estimators)

Name of the custom estimator must be unique.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for evaluate, version 0.0.3
Filename, size File type Python version Upload date Hashes
Filename, size evaluate-0.0.3.tar.gz (6.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page