alrelu-keras 0.1.0

ALReLU activation package for Keras with fixed and trainable alpha variants.

alrelu-keras

PyPI-installable Keras package for the ALReLU activation with 2 variants:

1. ALReLU (default): current behavior from this project, with fixed alpha=0.01.
2. TrainableALReLU: same formula with trainable alpha.

Formula used by both variants:

ALReLU(x, alpha) = max(abs(alpha * x), x)

Installation

After publishing:

pip install alrelu-keras

Keras requires a backend. For the common TensorFlow backend:

pip install alrelu-keras[tensorflow]

From source:

pip install .

Usage

1) Default ALReLU (current implementation behavior)

import keras
from alrelu_keras import ALReLU, alrelu

# As activation function
x = keras.ops.array([-2.0, -1.0, 0.0, 1.0], dtype="float32")
y = alrelu(x)  # alpha=0.01 by default

# As Keras layer
model = keras.Sequential(
    [
        keras.layers.Input(shape=(32,)),
        keras.layers.Dense(64),
        ALReLU(),  # fixed alpha
        keras.layers.Dense(10),
    ]
)

2) Trainable alpha variant

import keras
from alrelu_keras import TrainableALReLU

model = keras.Sequential(
    [
        keras.layers.Input(shape=(32,)),
        keras.layers.Dense(64),
        TrainableALReLU(alpha_initializer=0.01),  # alpha is trainable
        keras.layers.Dense(10),
    ]
)

Serialization / model loading

Both ALReLU, TrainableALReLU, and alrelu are registered as Keras-serializable objects. You can save/load models without manually passing custom_objects in standard Keras workflows.

Development

Install dev tools:

pip install -e .[dev]

Run tests:

pytest

Build package:

python -m build

License

MIT License. See LICENSE.