Expectation Reflection for classification
Project description
Expectation Reflection (ER) is a multiplicative optimization method that trains the interaction weights from features to target according to the ratio of target observations to their corresponding model expectations. This approach completely separates model updates from minimization of a cost function measuring goodness of fit, so that it can take the cost function as an effective stopping criterion of the iteration. This method has advantage in dealing with the problems of small sample sizes (but many features). Using only one hyper-parameter and being able to demonstrate the system mechanism are additional benefits of this method.
In the current version, the classification
package can work as a binary classifier. The extension to multiclass_classification
and regression
will be appeared shortly.
Installation
From PyPI
pip install expectation-reflection
From Repository
git clone https://github.com/danhtaihoang/expectation-reflection.git
Usage
The implementation of ER is very similar to that of other classifiers in sklearn
, bassically it consists of the following steps.
- Import the
expectation_reflection
package into your python script:
from expectation_reflection import classification as ER
- Select a model:
model = ER.model(max_iter=100,regu=0.01,random_state=1)
- Import your
dataset.txt
into python script. In the binary classification task, ER takes target of {0, 1} form:
Xy = np.loadtxt('dataset.txt')
- Select the features and target from the dataset. If the target is the last column then
X, y = Xy[:,:-1], Xy[:,-1]
- Import
train_test_split
fromsklearn
to split data into training and test sets:
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.5,random_state = 1)
- Train the model with
(X_train, y_train)
set:
model.fit(X_train, y_train)
- Predict the output class
y_pred
and its probabilityp_pred
of a new datasetX_test
:
y_pred = model.predict(X_test)
print('predicted output:', y_pred)
p_pred = model.predict_proba(X_test)
print('predicted probability:', p_pred)
- Intercept and interaction weights:
print('intercept:', model.intercept_)
print('interaction weights:', model.coef_)
Hyper-Parameter Optimization
ER has only one hyper-parameter, regu
, which can be optimized by using GridSearchCV
from sklearn
:
from sklearn.model_selection import GridSearchCV
model = ER.model(max_iter=100, random_state = 1)
regu = [0.0001, 0.001, 0.01, 0.1, 0.5, 1.]
hyper_parameters = dict(regu=regu)
clf = GridSearchCV(model, hyper_parameters=regu, cv=4, iid='deprecated')
best_model = clf.fit(X_train, y_train)
- Best hyper-parameters:
print('best_hyper_parameters:',best_model.best_params_)
- Predict the output
y_pred
and its probabilityp_pred
:
y_pred = best_model.best_estimator_.predict(X_test)
print('predicted output:', y_pred)
p_pred = best_model.best_estimator_.predict_proba(X_test)
print('predicted probability:', p_pred)
Performance Evaluation
We can measure the model performance by using metrics
from sklearn
:
from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score,\
roc_auc_score,roc_curve,auc
acc = accuracy_score(y_test,y_pred)
print('accuracy:', acc)
precision = precision_score(y_test,y_pred)
print('precision:', precision)
recall = recall_score(y_test,y_pred)
print('recall:', recall)
f1score = f1_score(y_test,y_pred)
print('f1score:', f1score)
roc_auc = roc_auc_score(y_test,p_pred) ## note: it is p_pred, not y_pred
print('roc auc:', roc_auc)
ROC AUC can be also calculated as
fp,tp,thresholds = roc_curve(y_test, p_pred, drop_intermediate=False)
roc_auc = auc(fp,tp)
print('roc auc:', roc_auc)
Citation
Please cite the following papers if you use this package in your work:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for expectation_reflection-0.0.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05a9d60460f4464433eece126cac4a6a4725a42c28023694ff7d9810a86200d7 |
|
MD5 | 941aa0f04797f4e110a43bb6642d9ef2 |
|
BLAKE2b-256 | 9c265be8eff77adc812dc69fdfda97bfca783a8fec3a007be63d8709ce3d9056 |
Hashes for expectation_reflection-0.0.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 500d85ee2216a0be5f756338825843eecaabacaa54c2ce8ccda7dc659cfa1d7d |
|
MD5 | 39bded2aafb521956d19f3622bb53038 |
|
BLAKE2b-256 | e116542ee42250cc4613a7d6855ae42f21f13657b83d07e58de79bb18436480a |