actix 0.4.0

A collection of novel and experimental activation functions for TensorFlow and PyTorch

Actix Functions

A collection of novel and experimental activation functions for deep learning, supporting both TensorFlow/Keras and PyTorch.

Key Features

• 50+ activation functions including custom parametric and static variants
• Dual framework support for TensorFlow/Keras and PyTorch
• Proven performance gains over standard activations in benchmarks
• Simple API with direct imports and getter functions

Installation

pip install actix                # Auto-detects installed frameworks
pip install actix[tf]            # TensorFlow only
pip install actix[torch]         # PyTorch only
pip install actix[tf,torch]      # Both frameworks

Quick Start

TensorFlow/Keras

from actix import OptimA, ATanSigmoid, get_activation
import tensorflow as tf

# Direct usage
model = tf.keras.Sequential([
    tf.keras.layers.Dense(128, input_shape=(784,)),
    OptimA(),
    tf.keras.layers.Dense(64),
    ATanSigmoid(),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Using getter function
activation = get_activation('OptimA', framework='tensorflow')

PyTorch

from actix import OptimATorch, ATanSigmoidTorch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 128)
        self.act1 = OptimATorch()
        self.fc2 = nn.Linear(128, 64)
        self.act2 = ATanSigmoidTorch()
        self.fc3 = nn.Linear(64, 10)
    
    def forward(self, x):
        x = self.act1(self.fc1(x))
        x = self.act2(self.fc2(x))
        return self.fc3(x)

Recommended Activations

Universal Top Performers

Function	Key Strength	Best Use Case
ATanSigU	Best overall performance	General-purpose, especially classification (77.63% on CIFAR-10)
A_ELuC	Consistent across tasks	When you need reliable performance everywhere

Classification Leaders

Rank	Activation Function	CIFAR-10 Accuracy
1	ATanSigU	77.63%
2	AdaptiveSinusoidalSoftgate	77.52%
3	ParametricLogish	77.48%
4	SmoothedAbsoluteGatedUnit	77.43%
5	A_ELuC	77.21%

Regression Champions

California Housing Dataset

Rank	Activation Function	MSE	Improvement vs Baseline
1	SymmetricParametricRationalSigmoid	0.2183	-17.75%
2	OptimA	0.2211	-16.70%
3	OptimXTemporal	0.2213	-16.62%

Diabetes Dataset

Rank	Activation Function	MSE	Improvement vs Baseline
1	ParametricBetaSoftsign	0.4564	-2.37%
2	A_ELuC	0.4567	-2.31%
3	SmoothedAbsoluteGatedUnit	0.4590	-1.82%

Experimental Functions

Function	Characteristics	Recommended For
ComplexHarmonicActivation	High potential, sensitive to hyperparameters	Advanced users, research projects
WeibullSoftplusActivation	Stable, predictable behavior	Production systems requiring reliability
GeneralizedAlphaSigmoid	Task-specific optimization	Specialized applications

Visualization Tools

import actix

# Visualize activation function and its derivative
actix.plot_activation('GeneralizedAlphaSigmoid', framework='tf')
actix.plot_derivative('GeneralizedAlphaSigmoid', framework='tf')

Requirements

• Python 3.7+
• NumPy ≥1.19
• Matplotlib ≥3.3
• TensorFlow ≥2.4 (optional)
• PyTorch ≥1.8 (optional)

Documentation

For complete function list and mathematical formulas, see actix/activations_tf.py

For detailed benchmarks, check:

• cifar.ipynb
• regression.ipynb

Contributing

We welcome contributions! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new functions
4. Ensure code follows PEP8 standards
5. Submit a pull request

License

MIT License - see LICENSE file for details.