Extra activation functions for Trax
Project description
Trax Extra Activation Functions
Extra activation functions for Trax
pip install trax-extra-activation
How to use
You can use activation functions in the same way as that ones implemented in Trax.
from trax_extra_activation import TanhExp
model = tl.Serial(
tl.Embedding(vocab_size=8192, d_feature=256),
tl.Mean(axis=1),
tl.Dense(2),
TanhExp()
)
Activations
- HardShrink
HardShrink(lmbda: float = 0.5) -> PureLayer
- HardSwish
HardSwish() -> PureLayer
- LogSigmoid
LogSigmoid() -> PureLayer
- Relu6
Relu6() -> PureLayer
- Celu
Celu(a: float = 1.0) -> PureLayer
- Silu
Silu() -> PureLayer
- Mish
Mish() -> PureLayer
- SoftShrink
SoftShrink(lmbda: float = 0.5) -> PureLayer
- SoftSign
SoftSign() -> PureLayer
- TanhShrink
TanhShrink() -> PureLayer
- TanhExp
TanhExp() -> PureLayer
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for trax-extra-activation-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd08352da2b0fed3799abb495488cca1ac1eef19543d3964ca3b5ded72676a4f |
|
MD5 | cc5f6acc2055682b7874fa7b451ff623 |
|
BLAKE2b-256 | a43fd909845f1c07f1c37152dfe8e3c194cf6ce687065d8e2ca31661c5d42f73 |
Close
Hashes for trax_extra_activation-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa9e625d72dc4f1185e8a8675c4d145bc0e89288eea5275bfe78e7b2aa461469 |
|
MD5 | 671c852f6cd9e34d6c7d2aa8d0f0d3ca |
|
BLAKE2b-256 | 4dddf4309b675535762b31ce78b32314efcb14c27fdc71b062b9873686a0b330 |