FWU
FWU is a series of PyTorch-based neural network units.It has higher information utilization, prevents neuronal necrosis, and eliminates the need to add additionall activation functions.
思路说明
ReLU滤除了小于零的输入,虽然高效实现了非线性,但对小于零的输入无梯度回传,导致了神经元坏死.目前有多种模仿ReLU的平滑版激活函数,通过在输入刚小于零时提供显著的反向梯度、在梯度更小时也提供微小的梯度,部分缓解了ReLU中神经元坏死的问题. FWU系列直接输入的符号将输入分为>0和<0两部分,分别送入两个单元(Linear或Conv或其他),再对两个单元的输出做二元运算(加法或乘法或其他).即提供了非线性,又最大程度上利用信息完全杜绝了神经元坏死.
目前提供了加法(FWLU)和乘法(FWMU)的Linear、Conv1d、Conv2d、和对应转置卷积的实现.无后缀代表Linear,1D、2D后缀代表对应维度的卷积,加T后缀代表转置卷积.
部分参数,如kernel_size、padding可以通过加_N后缀实现对负分支的专门控制.用以对正负分支添加不同的前后偏移或使用不同宽高比的卷积核,可以减少参数量增大感受野.
Install
pip install FWU
Use
from FWU import FWLU2DT
...
import后直接替代对应的Linear、Conv1d、Conv2d等,并取消与其组合的激活函数(输出层的SoftMax等承担特殊功能的激活函数除外).
HomePage
1 maintainer
Metadata
1 maintainer
Release files for FWU 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| fwu-0.0.4.tar.gz | 11.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| fwu-0.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 26.5 kB
Release files / fwu-0.0.4.tar.gz
| Download URL | fwu-0.0.4.tar.gz |
|---|---|
| Size | 11.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
96d35e39b15f93eca45c3fab1789b4e08b9fed6c128010f3372a96e7f5ca9e75
|
|
BLAKE2b-256 checksum How to use checksums |
f42292ad37df2d4931ef78b0247cc4f68ea611d4f87f82b0b20b5b7396026a3a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.12.9
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 29, 2025.
Transparency logRelease files / fwu-0.0.4-py3-none-any.whl
| Download URL | fwu-0.0.4-py3-none-any.whl |
|---|---|
| Size | 15.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7e18876dd63ae16f66905a5bca37f3465b8e20c69f8c526b6226d28ea2a3bfc7
|
|
BLAKE2b-256 checksum How to use checksums |
dc3fccd64783001f166c73f4a6bf46583dca1da813251a2ca531c459c9c27648
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.12.9
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 29, 2025.
Transparency log
