Actix Functions

A collection of novel and experimental activation functions for deep learning, supporting both TensorFlow/Keras and PyTorch.

Key Features

• 50+ activation functions including custom parametric and static variants
• Dual framework support for TensorFlow/Keras and PyTorch
• Proven performance gains over standard activations in benchmarks
• Simple API with direct imports and getter functions

Installation

pip install actix                # Auto-detects installed frameworks
pip install actix[tf]            # TensorFlow only
pip install actix[torch]         # PyTorch only
pip install actix[tf,torch]      # Both frameworks

Quick Start

TensorFlow/Keras

from actix import OptimA, ATanSigmoid, get_activation
import tensorflow as tf

# Direct usage
model = tf.keras.Sequential([
    tf.keras.layers.Dense(128, input_shape=(784,)),
    OptimA(),
    tf.keras.layers.Dense(64),
    ATanSigmoid(),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Using getter function
activation = get_activation('OptimA', framework='tensorflow')

PyTorch

from actix import OptimATorch, ATanSigmoidTorch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 128)
        self.act1 = OptimATorch()
        self.fc2 = nn.Linear(128, 64)
        self.act2 = ATanSigmoidTorch()
        self.fc3 = nn.Linear(64, 10)
    
    def forward(self, x):
        x = self.act1(self.fc1(x))
        x = self.act2(self.fc2(x))
        return self.fc3(x)

Recommended Activations

Universal Top Performers

Classification Leaders

Regression Champions

California Housing Dataset

Diabetes Dataset

Experimental Functions

Visualization Tools

import actix

# Visualize activation function and its derivative
actix.plot_activation('GeneralizedAlphaSigmoid', framework='tf')
actix.plot_derivative('GeneralizedAlphaSigmoid', framework='tf')

Requirements

• Python 3.7+
• NumPy ≥1.19
• Matplotlib ≥3.3
• TensorFlow ≥2.4 (optional)
• PyTorch ≥1.8 (optional)

Documentation

For complete function list and mathematical formulas, see actix/activations_tf.py

For detailed benchmarks, check:

• cifar.ipynb
• regression.ipynb

Contributing

We welcome contributions! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new functions
4. Ensure code follows PEP8 standards
5. Submit a pull request

License

MIT License - see LICENSE file for details.