META Learn

MetaRL-based Estimator using Task-encodings for Automated machine Learning

META Learn is a deep learning approach to automated machine learning that parameterizes the API of machine learning software as a sequence of actions to select the hyperparameters of a machine learning pipeline in an end-to-end fashion, from raw data representation, imputation, normalizing, feature representation, and classification/regression. Currently the sklearn API is the only supported ML framework.

Why?

As the diversity of data and machine learning use cases increases, we need to accelerate and scale the process of training performant machine learning systems. We'll need tools that are adaptable to specific problem domains, datasets, and the (sometimes non-differentiable) performance metrics that we're trying to optimize. Supervised learning of classification and regression tasks given a task distribution of small to medium datasets provides an promising jumping off platform for programmatically generating a reinforcement learning environment for automated machine learning (AutoML).

Installation

install metalearn library:

pip install -e .

then you can run an experiment with the metalearn cli.

# run an experiment with default values
$ metalearn run experiment

Running an experiment with a configuration file

Alternatively, you can create an experiment configuration file to run your experiment.

# create experiment config file
$ metalearn create config my_experiment config/local --description "my experiment"

# output:
# wrote experiment config file to config/local/experiment_2018-37-25-21:37:11_my_experiment.yml

edit the config file parameters section to the set of parameters that you want to train on, then run the experiment with

$ metalearn run from-config config/local/experiment_2018-37-25-21:37:11_my_experiment.yml

run an experiment using floydhub:

$ floyd_scripts/train-metalearn.sh path/to/config/file 'optional job run message'

Relevant Work

The Combined Algorithm Selection and Hyperparameter optimization (CASH) problem is an important one to solve if we want to effectively scale and deploy machine learning systems in real-world use cases, which often deals with small (< 10 gb) to medium size (10 - 100 gb) data.

CASH is the problem of searching through the space of all ML frameworks, defined as an Algorithm A and a set of relevant hyperparameters lambda and proposing a set of models that will perform well given a dataset and a task.

In order to solve this problem, previous work like autosklearn uses a Bayesian Optimization techniques SMAC with an offline meta- learning "warm-start" step using euclidean distance to reduce the search space of ML frameworks. This meta-learning step was done by representing the datasets with metadata features (e.g. number of features, skew, mean, variance, etc.) to learn representations of the data space that perform well with respect to the ML framework selection task.

Neural Architecture Search is another approach to the CASH problem, where a Controller network proposes "child" neural net architectures that are trained on a training set and evaluated on a validation set, using the validation performance R as a reinforcement learning reward signal to learn the best architecture proposal policy.

Contributions

The contributions of the META Learn project are two-fold: it builds on the neural architecture search paradigm by formalating the output space of the Controller as a sequence of tokens conditioned on the space of possible executable frameworks. The scope of this project is to define a framework, expressed as a set of hyperparameters, that can be evaluated by a machine learning framework, like sklearn, which evaluates to an instantiated sklearn Pipeline. Once defined, it can be fitted on a training set and evaluated on a validation set of a particular dataset D.

Following the Neural Architecture scheme, META Learn uses the REINFORCE algorithm to compute the policy gradient used to update the Controller in order to learn a policy for proposing good frameworks that are able to achieve high validation set performance.

The second contribution of this project is that it proposes a conditional ML framework generator by extending the Controller network to have an encoder network that takes as input metadata about the dataset D (e.g. number of instances, number of features). The output of the encoder network would be fed into the decoder network, which proposes an ML framework. Therefore, we can condition the output of the decoder network metadata on D to propose frameworks that depend on the encoded dataset representation.

Training Algorithm

The environment is a distribution of k supervised learning tasks, consisting of a pair (X, y) of features and targets, respectively. At the beginning of an episode, the environment samples one task and for i iterations produces sample splits (X_train, y_train, X_validation, y_validation) drawn from the task dataset.

The MetaLearnController receives the current task state s via metafeatures associated the task, e.g. # of training samples, # of features, target type, #of continuous features, #of categorical features, etc.

Given the task state, the controller generates ML frameworks over a state space of algorithms and hyperparameter values. The controller can be viewed as a policy approximator, which selects actions based on some pre-defined AlgorithmSpace, representated as a direct acyclic graph where each node contains a set of hyperparameter values to choose from. The controller traverses this graph via a sequential decoder by selecting hyperparameters via softmax classifiers, where certain actions may remove certain edges from the graph. This enforces incompatible hyperparameter configurations.

For example, the algorithm space for a possible sklearn pipeline would consist of the following components:

• categorical encoder (e.g. OneHotEncoder)
• categorical encoder hyperparameters
• imputer (e.g. SimpleImputer)
• imputer hyperparameters
• rescaler (e.g. StandardScaler)
• rescaler hyperparameters
• feature processor (e.g. PCA)
• feature processor hyperparameters
• classifier/regressor (e.g. LogisticRegression, LinearRegression)
• classifier/regressor hyperparameters

When the controller reaches a terminal node in algorithm space, the environment evalutes the selected ML framework and produces a validation score that the controller uses as a reward signal. Validation performance is calibrated such that a better score produces higher rewards. Using the REINFORCE policy gradient method, the controller tries to find the optimal policy that maximizes validation performance over the task distribution.

Analyses

The ./analysis subfolder contains jupyter notebooks that visualize the performance of the cash controller over time. Currently there are 5 analyses in the project analysis subfolder:

• 20180715_metalearn_controller_analysis.ipynb: a basic interactive analysis of a single job's outputs.
• 20180809_experiments_jobs_150_151.ipynb: analyzes baseline performance based on correct action selection implementation. For more info on the bug, see here.
• 20180902_experiments_entropy_jobs_166-175.ipynb: analyzes tuning experiment after entropy coefficient was introduced in the loss function.
• 20180930_experiments_regression_195.ipynb: analyzes a pure regression metalearn controller.
• 20181003_experiments_regression_197.ipynb: analyzes a pure regression controller with two sets of entropy coef settings.
• 20191111_metalearn_openml_cc18_benchmark.ipynb: evaluate the metalearn controller's ability to learn from an internal RL algorithm implemented within the RNN loop.