Pipelines and primitives for machine learning and data science.
Project description
An open source project from Data to AI Lab at MIT.
MLPrimitives
Pipelines and primitives for machine learning and data science.
- Free software: MIT license
- Documentation: https://HDI-Project.github.io/MLPrimitives
Overview
This repository contains primitive annotations to be used by the MLBlocks library, as well as the necessary Python code to make some of them fully compatible with the MLBlocks API requirements.
There is also a collection of custom primitives contributed directly to this library, which either combine third party tools or implement new functionalities from scratch.
Why did we create this library?
- Too many libraries in a fast growing field
- Huge societal need to build machine learning apps
- Domain expertise resides at several places (knowledge of math)
- No documented information about hyperparameters, behavior...
Installation
Requirements
MLPrimitives has been developed and tested on Python 3.5, and 3.6
Also, although it is not strictly required, the usage of a virtualenv is highly recommended in order to avoid interfering with other software installed in the system where MLPrimitives is run.
These are the minimum commands needed to create a virtualenv using python3.6 for MLPrimitives:
pip install virtualenv
virtualenv -p $(which python3.6) mlprimitives-venv
Afterwards, you have to execute this command to have the virtualenv activated:
source mlprimitives-venv/bin/activate
Remember about executing it every time you start a new console to work on MLPrimitives!
Install using Pip
After creating the virtualenv and activating it, we recommend using pip in order to install MLPrimitives:
pip install mlprimitives
This will pull and install the latest stable release from PyPi.
Install from Source
Alternatively, with your virtualenv activated, you can clone the repository and install it from
source by running make install
on the stable
branch:
git clone git@github.com:HDI-Project/MLPrimitives.git
cd MLPrimitives
git checkout stable
make install
Install for Development
If you want to contribute to the project, a few more steps are required to make the project ready for development.
First, please head to the GitHub page of the project and make a fork of the project under you own username by clicking on the fork button on the upper right corner of the page.
Afterwards, clone your fork and create a branch from master with a descriptive name that includes the number of the issue that you are going to work on:
git clone git@github.com:{your username}/MLPrimitives.git
cd MLPrimitives
git branch issue-xx-cool-new-feature master
git checkout issue-xx-cool-new-feature
Finally, install the project with the following command, which will install some additional dependencies for code linting and testing.
make install-develop
Make sure to use them regularly while developing by running the commands make lint
and make test
.
Quickstart
This section is a short series of tutorials to help you getting started with MLPrimitives.
In the following steps you will learn how to load and run a primitive on some data.
Later on you will learn how to evaluate and improve the performance of a primitive by tuning its hyperparameters.
Running a Primitive
In this first tutorial, we will be executing a single primitive for data transformation.
1. Load a Primitive
The first step in order to run a primitive is to load it.
This will be done using the mlprimitives.load_primitive
function, which will
load the indicated primitive as an MLBlock Object from MLBlocks
In this case, we will load the mlprimitives.custom.feature_extraction.CategoricalEncoder
primitive.
from mlprimitives import load_primitive
primitive = load_primitive('mlprimitives.custom.feature_extraction.CategoricalEncoder')
2. Load some data
The CategoricalEncoder is a transformation primitive which applies one-hot encoding to all the
categorical columns of a pandas.DataFrame
.
So, in order to be able to run our primitive, we will first load some data that contains categorical columns.
This can be done with the mlprimitives.datasets.load_census
function:
from mlprimitives.datasets import load_census
dataset = load_census()
This dataset object has an attribute data
which contains a table with several categorical
columns.
We can have a look at this table by executing dataset.data.head()
, which will return a
table like this:
0 1 2
age 39 50 38
workclass State-gov Self-emp-not-inc Private
fnlwgt 77516 83311 215646
education Bachelors Bachelors HS-grad
education-num 13 13 9
marital-status Never-married Married-civ-spouse Divorced
occupation Adm-clerical Exec-managerial Handlers-cleaners
relationship Not-in-family Husband Not-in-family
race White White White
sex Male Male Male
capital-gain 2174 0 0
capital-loss 0 0 0
hours-per-week 40 13 40
native-country United-States United-States United-States
3. Fit the primitive
In order to run our pipeline, we first need to fit it.
This is the process where it analyzes the data to detect which columns are categorical
This is done by calling its fit
method and assing the dataset.data
as X
.
primitive.fit(X=dataset.data)
4. Produce results
Once the pipeline is fit, we can process the data by calling the produce
method of the
primitive instance and passing agin the data
as X
.
transformed = primitive.produce(X=dataset.data)
After this is done, we can see how the transformed data contains the newly generated one-hot vectors:
0 1 2 3 4
age 39 50 38 53 28
fnlwgt 77516 83311 215646 234721 338409
education-num 13 13 9 7 13
capital-gain 2174 0 0 0 0
capital-loss 0 0 0 0 0
hours-per-week 40 13 40 40 40
workclass= Private 0 0 1 1 1
workclass= Self-emp-not-inc 0 1 0 0 0
workclass= Local-gov 0 0 0 0 0
workclass= ? 0 0 0 0 0
workclass= State-gov 1 0 0 0 0
workclass= Self-emp-inc 0 0 0 0 0
... ... ... ... ... ...
Tuning a Primitive
In this short tutorial we will teach you how to evaluate the performance of a primitive and improve its performance by modifying its hyperparameters.
To do so, we will load a primitive that can learn from the transformed data that we just generated and later on make predictions based on new data.
1. Load another primitive
Firs of all, we will load the xgboost.XGBClassifier
primitive that we will use afterwards.
primitive = load_primitive('xgboost.XGBClassifier')
2. Split the dataset
Before being able to evaluate the primitive perfomance, we need to split the data in two parts: train, which will be used for the primitive to learn, and test, which will be used to make the predictions that later on will be evaluated.
In order to do this, we will get the first 75% of rows from the transformed data that we
obtained above and call it X_train
, and then set the next 25% of rows as X_test
.
train_size = int(len(transformed) * 0.75)
X_train = transformed.iloc[:train_size]
X_test = transformed.iloc[train_size:]
Similarly, we need to obtain the y_train
and y_test
variables containing the corresponding
output values.
y_train = dataset.target[:train_size]
y_test = dataset.target[train_size:]
3. Fit the new primitive
Once we have have splitted the data, we can fit the primitive by passing X_train
and y_train
to its fit
method.
primitive.fit(X=X_train, y=y_train)
4. Make predictions
Once the primitive has been fitted, we can produce predictions using the X_test
data as input.
predictions = primitive.produce(X=X_test)
5. Evalute the performance
We can now evaluate how good the predictions from our primitive are by using the score
method from the dataset
object on both the expected output and the real output from the
primitive:
dataset.score(y_test, predictions)
This will output a float value between 0 and 1 indicating how good the predicitons are, being 0 the worst score possible and 1 the best one.
In this case we will obtain a score around 0.866
6. Set new hyperparameter values
In order to improve the performance of our primitive we will try to modify a couple of its hyperparameters.
First we will see which hyperparameter values the primitive has by calling its
get_hyperparameters
method.
primitive.get_hyperparameters()
which will return a dictionary like this:
{
"n_jobs": -1,
"n_estimators": 100,
"max_depth": 3,
"learning_rate": 0.1,
"gamma": 0,
"min_child_weight": 1
}
Next, we will see which are the valid values for each one of those hyperparameters by calling its
get_tunable_hyperparameters
method:
primitive.get_tunable_hyperparameters()
For example, we will see that the max_depth
hyperparameter has the following specification:
{
"type": "int",
"default": 3,
"range": [
3,
10
]
}
Next, we will choose a valid value, for example 7, and set it into the pipeline using the
set_hyperparameters
method:
primitive.set_hyperparameters({'max_depth': 7})
7. Re-evaluate the performance
Once the new hyperparameter value has been set, we repeat the fit/train/score cycle to evaluate the performance of this new hyperparameter value:
primitive.fit(X=X_train, y=y_train)
predictions = primitive.produce(X=X_test)
dataset.score(y_test, predictions)
This time we should see that the performance has improved to a value around 0.724
What's Next?
Do you want to learn more about how the project, about how to contribute to it or browse the API Reference? Please check the corresponding sections of the documentation!
History
0.2.3 - 2019-11-14
New Primitives
Add primitive to make window_sequences based on cutoff times - Issue #217 by @csala Create a keras LSTM based TimeSeriesClassifier primitive - Issue #218 by @csala Add pandas DataFrame primitives - Issue #214 by @csala Add featuretools.EntitySet.normalize_entity primitive - Issue #209 by @csala
Primitive Improvements
Make featuretools.EntitySet.entity_from_dataframe entityset arg optional - Issue #208 by @csala
Add text regression dataset - Issue #206 by @csala
Bug Fixes
pandas.DataFrame.resample crash when grouping by integer columns - Issue #211 by @csala
0.2.2 - 2019-10-08
New Primitives
- Add primitives for GAN based time-series anomaly detection - Issue #200 by @AlexanderGeiger
- Add
numpy.reshape
andnumpy.ravel
primitives - Issue #197 by @AlexanderGeiger - Add feature selection primitive based on Lasso - Issue #194 by @csala
Primitive Improvements
feature_extraction.CategoricalEncoder
support dtype category - Issue #196 by @csala
0.2.1 - 2019-09-09
New Primitives
- Timeseries Intervals to Mask Primitive - Issue #186 by @AlexanderGeiger
- Add new primitive: Arima model - Issue #168 by @AlexanderGeiger
Primitive Improvements
- Curate PCA primitive hyperparameters - Issue #190 by @AlexanderGeiger
- Add option to drop rolling window sequences - Issue #186 by @AlexanderGeiger
Bug Fixes
- scikit-image==0.14.3 crashes when installed on Mac - Issue #188 by @csala
0.2.0
New Features
- Publish the pipelines as an
entry_point
Issue #175 by @csala
Primitive Improvements
- Improve pandas.DataFrame.resample primitive Issue #177 by @csala
- Improve
feature_extractor
primitives Issue #183 by @csala - Improve
find_anomalies
primitive Issue #180 by @AlexanderGeiger
Bug Fixes
- Typo in the primitive keras.Sequential.LSTMTimeSeriesRegressor Issue #176 by @DanielCalvoCerezo
0.1.10
New Features
- Add function to run primitives without a pipeline Issue #43 by @csala
New Pipelines
- Add pipelines for all the MLBlocks examples Issue #162 by @csala
Primitive Improvements
- Add Early Stopping to
keras.Sequential.LSTMTimeSeriesRegressor
primitive Issue #156 by @csala - Make FeatureExtractor primitives accept Numpy arrays Issue #165 by @csala
- Add window size and pruning to the
timeseries_anomalies.find_anomalies
primitive Issue #160 by @csala
0.1.9
New Features
- Add a single table binary classification dataset Issue #141 by @csala
New Primitives
- Add Multilayer Perceptron (MLP) primitive for binary classification Issue #140 by @Hector-hedb12
- Add primitive for Sequence classification with LSTM Issue #150 by @Hector-hedb12
- Add VGG-like convnet primitive Issue #149 by @Hector-hedb12
- Add Multilayer Perceptron (MLP) primitive for multi-class softmax classification Issue #139 by @Hector-hedb12
- Add primitive to count feature matrix columns Issue #146 by @csala
Primitive Improvements
- Add additional fit and predict arguments to keras.Sequential Issue #161 by @csala
- Add suport for keras.Sequential Callbacks Issue #159 by @csala
- Add fixed hyperparam to control keras.Sequential verbosity Issue #143 by @csala
0.1.8
New Primitives
- mlprimitives.custom.timeseries_preprocessing.time_segments_average - Issue #137
New Features
- Add target_index output in timseries_preprocessing.rolling_window_sequences - Issue #136
0.1.7
General Improvements
- Validate JSON format in
make lint
- Issue #133 - Add demo datasets - Issue #131
- Improve featuretools.dfs primitive - Issue #127
New Primitives
- pandas.DataFrame.resample - Issue #123
- pandas.DataFrame.unstack - Issue #124
- featuretools.EntitySet.add_relationship - Issue #126
- featuretools.EntitySet.entity_from_dataframe - Issue #126
Bug Fixes
- Bug in timeseries_anomalies.py - Issue #119
0.1.6
General Improvements
- Add Contributing Documentation
- Remove upper bound in pandas version given new release of
featuretools
v0.6.1 - Improve LSTMTimeSeriesRegressor hyperparameters
New Primitives
- mlprimitives.candidates.dsp.SpectralMask
- mlprimitives.custom.timeseries_anomalies.find_anomalies
- mlprimitives.custom.timeseries_anomalies.regression_errors
- mlprimitives.custom.timeseries_preprocessing.rolling_window_sequences
- mlprimitives.custom.timeseries_preprocessing.time_segments_average
- sklearn.linear_model.ElasticNet
- sklearn.linear_model.Lars
- sklearn.linear_model.Lasso
- sklearn.linear_model.MultiTaskLasso
- sklearn.linear_model.Ridge
0.1.5
New Primitives
- sklearn.impute.SimpleImputer
- sklearn.preprocessing.MinMaxScaler
- sklearn.preprocessing.MaxAbsScaler
- sklearn.preprocessing.RobustScaler
- sklearn.linear_model.LinearRegression
General Improvements
- Separate curated from candidate primitives
- Setup
entry_points
in setup.py to improve compaitibility with MLBlocks - Add a test-pipelines command to test all the existing pipelines
- Clean sklearn example pipelines
- Change the
author
entry to acontributors
list - Change the name of
mlblocks_primitives
folder - Pip install
requirements_dev.txt
fail documentation
Bug Fixes
- Fix LSTMTimeSeriesRegressor primitive. Issue #90
- Fix timeseries primitives. Issue #91
- Negative index anomalies in
timeseries_errors
. Issue #89 - Keep pandas version below 0.24.0. Issue #87
0.1.4
New Primitives
- mlprimitives.timeseries primitives for timeseries data preprocessing
- mlprimitives.timeseres_error primitives for timeseries anomaly detection
- keras.Sequential.LSTMTimeSeriesRegressor
- sklearn.neighbors.KNeighbors Classifier and Regressor
- several sklearn.decomposition primitives
- several sklearn.ensemble primitives
Bug Fixes
- Fix typo in mlprimitives.text.TextCleaner primitive
- Fix bug in index handling in featuretools.dfs primitive
- Fix bug in SingleLayerCNNImageClassifier annotation
- Remove old vlaidation tags from JSON annotations
0.1.3
New Features
- Fix and re-enable featuretools.dfs primitive.
0.1.2
New Features
- Add pipeline specification language and Evaluation utilities.
- Add pipelines for graph, text and tabular problems.
- New primitives ClassEncoder and ClassDecoder
- New primitives UniqueCounter and VocabularyCounter
Bug Fixes
- Fix TrivialPredictor bug when working with numpy arrays
- Change XGB default learning rate and number of estimators
0.1.1
New Features
- Add more keras.applications primitives.
- Add a Text Cleanup primitive.
Bug Fixes
- Add keywords to
keras.preprocessing
primtives. - Fix the
image_transform
method. - Add
epoch
as a fixed hyperparameter forkeras.Sequential
primitives.
0.1.0
- First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mlprimitives-0.2.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87ed05b35088e9b090914f9117e31e07f887bf9af4c7923c17fed41f6366aaaa |
|
MD5 | b4a3dbff80f0e969d91c3d1704372be2 |
|
BLAKE2b-256 | f43f5694356187061867e41390ad33e8fc3c6d92cf1f712cc0165aae8136349f |