Expectation Reflection for classification
Project description
Expectation Reflection (ER) is a multiplicative optimization method that trains the interaction weights from features to target according to the ratio of target observations to their corresponding model expectations. This approach completely separates model updates from minimization of a cost function measuring goodness of fit, so that it can take the cost function as an effective stopping criterion of the iteration.
Advantages of this method: (1) working relatively well even in the regime of small sample sizes; (2) using only one hyperparameter; (3) being able to demonstrate the system mechanism.
In the current version, ER classification
can work as a classifier (for both binary and multinomial tasks). The extension to regression
will be appeared shortly.
Installation
From PyPI
pip install expectationreflection
From Repository
git clone https://github.com/danhtaihoang/expectationreflection.git
Usage
The implementation of ER is very similar to that of other classifiers in sklearn
, bassically it consists of the following steps.
 Import the
expectation_reflection
package into your python script:
from expectation_reflection import classification as ER
 Select a model:
model = ER.model(max_iter=100,reg=0.01,random_state=1)
 Import your
dataset.txt
into python script.
Xy = np.loadtxt('dataset.txt')
 Select the features and target from the dataset. If the target is the last column then
X, y = Xy[:,:1], Xy[:,1]
 Import
train_test_split
fromsklearn
to split data into training and test sets:
from sklearn.model_selection import train_test_split X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.5,random_state = 1)
 Train the model with
(X_train, y_train)
set:
model.fit(X_train, y_train)
 Predict the output class
y_pred
and its probabilityp_pred
of a new datasetX_test
:
y_pred = model.predict(X_test) print('predicted output:', y_pred) p_pred = model.predict_proba(X_test) print('predicted probability:', p_pred)
 Intercept and interaction weights:
print('intercept:', model.intercept_) print('interaction weights:', model.coef_)
HyperParameter Optimization
ER has only one hyperparameter, reg
, which can be optimized by using GridSearchCV
from sklearn
:
from sklearn.model_selection import GridSearchCV model = ER.model(max_iter=100, random_state = 1) reg = [0.0001, 0.001, 0.01, 0.1, 0.5, 1.] hyper_parameters = dict(reg=reg) clf = GridSearchCV(model, hyper_parameters, cv=4, n_jobs=1, iid='deprecated') best_model = clf.fit(X_train, y_train)
 Best hyperparameters:
print('best_hyper_parameters:',best_model.best_params_)
 Predict the output
y_pred
and its probabilityp_pred
:
y_pred = best_model.best_estimator_.predict(X_test) print('predicted output:', y_pred) p_pred = best_model.best_estimator_.predict_proba(X_test) print('predicted probability:', p_pred)
Performance Evaluation
We can measure the model performance by using metrics
from sklearn
:
from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score,\ roc_auc_score,roc_curve,auc acc = accuracy_score(y_test,y_pred) print('accuracy:', acc) precision = precision_score(y_test,y_pred) print('precision:', precision) recall = recall_score(y_test,y_pred) print('recall:', recall) f1score = f1_score(y_test,y_pred) print('f1score:', f1score) roc_auc = roc_auc_score(y_test,p_pred) ## note: it is p_pred, not y_pred print('roc auc:', roc_auc)
ROC AUC can be also calculated as
fp,tp,thresholds = roc_curve(y_test, p_pred, drop_intermediate=False) roc_auc = auc(fp,tp) print('roc auc:', roc_auc)
Citation
Please cite the following papers if you use this package in your work:
Project details
Release history Release notifications  RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for expectation_reflection0.0.10.tar.gz
Algorithm  Hash digest  

SHA256  f0b181df845555bced4e73b11bbd02f5b3007cf7df80230398a6953faced0f44 

MD5  6a69527f1bb0395796e4e46c030b1cf3 

BLAKE2256  b3b2d206d789930a0fed49a860db8cb8f47c6d999e10ba386216b3aba6ba5771 
Hashes for expectation_reflection0.0.10py3noneany.whl
Algorithm  Hash digest  

SHA256  2167823e1ae45b390de15d5a7912dac51b5cd8bda230f9ae614a9c139a554394 

MD5  0f50820bac3446572686184d3d45ea3d 

BLAKE2256  daa80bf168e97d57fc0f284f73e70287a6e535b5af2ff10afa8133fe00f9adeb 