Activations Plus
Activations Plus is a Python package designed to provide a collection of advanced activation functions for machine learning and deep learning models. These activation functions are implemented to enhance the performance of neural networks by addressing specific challenges such as sparsity, non-linearity, and gradient flow.
Features
- Entmax: Sparse activation function for probabilistic models.
- Sparsemax: Sparse alternative to softmax.
- Bent Identity: Smooth approximation of the identity function. (Experimental feature require review)
- ELiSH (Exponential Linear Squared Hyperbolic): Combines exponential and linear properties. (Experimental feature require review)
- Maxout: Learns piecewise linear functions. (Experimental feature require review)
- Soft Clipping: Smoothly clips values to a range. (Experimental feature require review)
- SReLU (S-shaped Rectified Linear Unit): Combines linear and non-linear properties. (Experimental feature require review)
Installation
To install the package, use pip:
pip install activations-plus
Usage
Import and use any activation function in your PyTorch models:
import torch
from activations_plus.sparsemax import Sparsemax
from activations_plus.entmax import Entmax
# Example with Sparsemax
sparsemax = Sparsemax()
x = torch.tensor([[1.0, 2.0, 3.0], [1.0, 2.0, -1.0]])
output_sparsemax = sparsemax(x)
print("Sparsemax Output:", output_sparsemax)
# Example with Entmax
entmax = Entmax(alpha=1.5)
output_entmax = entmax(x)
print("Entmax Output:", output_entmax)
These examples demonstrate how to use Sparsemax and Entmax activation functions in PyTorch models.
Documentation
Comprehensive documentation is available documentation.
Supported Activation Functions
- Entmax: Sparse activation function for probabilistic models. Reference Paper
- Sparsemax: Sparse alternative to softmax for probabilistic outputs. Reference Paper
- Bent Identity: A smooth approximation of the identity function. (Experimental feature require review) reference missing
- ELiSH: Combines exponential and linear properties for better gradient flow. (Experimental feature require review) Reference Paper
- Maxout: Learns piecewise linear functions for better expressiveness. (Experimental feature require review) Reference Paper
- Soft Clipping: Smoothly clips values to a range to avoid extreme outputs. (Experimental feature require review) Reference Paper
- SReLU: Combines linear and non-linear properties for better flexibility. (Experimental feature require review) Reference Paper
Contributing
Contributions are welcome! Please read the CONTRIBUTING.md file for guidelines.
Testing
To run the tests, use the following command:
pytest tests/
License
This project is licensed under the MIT License. See the LICENSE file for details.
Acknowledgments
Special thanks to the contributors and the open-source community for their support.
Metadata
Release files for activations-plus 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| activations_plus-0.1.1.tar.gz | 10.4 kB | Without provenance | Details for activations_plus-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| activations_plus-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for activations_plus-0.1.1-py3-none-any.whl |
Total release size: 26.3 kB
Release files / activations_plus-0.1.1.tar.gz
| Download URL | activations_plus-0.1.1.tar.gz |
|---|---|
| Size | 10.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
35bf8cf8da08da571b5c065e980bcf3bb91e5529499045c1cee7c0dbd84bfaff
|
|
BLAKE2b-256 checksum How to use checksums |
e3e24be2a801e73a0ee6364fd827ff4e727aee1a76c970cb07d502093fbd531f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.14
|
Release files / activations_plus-0.1.1-py3-none-any.whl
| Download URL | activations_plus-0.1.1-py3-none-any.whl |
|---|---|
| Size | 15.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
96e43974052dc0dd809732883cd2b0a7954cf49132a8517795146a56a59dcb62
|
|
BLAKE2b-256 checksum How to use checksums |
f41fff0932d2cb8d6a799d3afced643a2aea1a9106e1fe5f4df9748e1dfefcf5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.14
|
Security for activations-plus 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.