Keras Targeted Dropout

Unofficial implementation of Targeted Dropout with tensorflow backend. Note that there is no model compression in this implementation.

Install

pip install keras-targeted-dropout

Usage

import keras
from keras_targeted_dropout import TargetedDropout

model = keras.models.Sequential()
model.add(TargetedDropout(
    layer=keras.layers.Dense(units=2, activation='softmax'),
    drop_rate=0.8,
    target_rate=0.2,
    drop_patterns=['kernel'],
    mode=TargetedDropout.MODE_UNIT,
    input_shape=(5,),
))
model.compile(optimizer='adam', loss='mse')
model.summary()

• drop_rate: Dropout rate for each pixel.
• target_rate: The proportion of bottom weights selected as candidates
• drop_patterns: A list of names of weights to be dropped.
• mode: TargetedDropout.MODE_UNIT or TargetedDropout.MODE_WEIGHT.

The final dropout rate will be drop_rate times target_rate.