Afine Optimizer
An optimizer that is superior to Adam. Based on @PyTorch
The training is more stable and reaches a more elegant optimal solution faster.
思路说明
把每组参数看作高维向量. 优化步骤被分割为3个过程: 旋转、缩放、平移. 使用三个学习率分别控制三个过程的调整量.
1.旋转
- 计算梯度对参数向量方向的影响.
- 根据旋转的稳定程度调整旋转的作用强度.
- 使用近似旋转改变参数向量方向而不改变模长.
2.缩放
- 计算梯度对参数向量模长的影响.
- 根据模长改变的稳定程度调整缩放的作用强度.
- 权重衰减作用于这一步: 使用ArcSinh函数对向量放大带来阻力而促进向量缩小.
- 应用缩放变换.
3.平移
- 旋转和缩放对零向量无作用, 为使零初始化和模长较小的向量也能被训练, 需加入平移.
- 此处的平移带有惯性,且平移量与梯度稳定程度有关.
Demo程序
demo程序在test.py文件中,直接运行即可得到此页面中Affine与RAdam的对比图.(依赖visdom) demo中构建了一个神经网络用简陋的方式对一个包含动态随机参数的公式生成的周期性曲线进行预测.
Install
pip install afine
Use
from Afine import Afine
...
optimizer=Affine(moduel.parameters(),lr_rotate = lr_rotate,lr_scale = lr_scale,lr_base = lr_base,weight_decay=weight_decay)
HomePage
1 maintainer
Metadata
1 maintainer
Release files for Afine 0.0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| afine-0.0.2.tar.gz | 11.1 kB | With provenance | Details for afine-0.0.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| Afine-0.0.2-py3-none-any.whl | Python 3 | none | any | Details for Afine-0.0.2-py3-none-any.whl |
Total release size: 22.5 kB
Release files / afine-0.0.2.tar.gz
| Download URL | afine-0.0.2.tar.gz |
|---|---|
| Size | 11.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e1357e4e9483e80f35c3bee6daa1ef6ab488656391671a50d0691063f6a0b4aa
|
|
BLAKE2b-256 checksum How to use checksums |
aa547cbb388146f4d4aceda6a2285e8ab8e1f58556996c86b21ccb6b055ad23e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.0.1 CPython/3.12.8
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 17, 2025.
Transparency logRelease files / Afine-0.0.2-py3-none-any.whl
| Download URL | Afine-0.0.2-py3-none-any.whl |
|---|---|
| Size | 11.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
651a8dd60ccc5362c60624577e6829b92750d0e31bf2087fb3247651be806ee5
|
|
BLAKE2b-256 checksum How to use checksums |
2253220ec07932588d5392f0bd6ffa69fc725863d34e30cb91eceaa193778af8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.0.1 CPython/3.12.8
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 17, 2025.
Transparency logSecurity for Afine 0.0.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for afine-0.0.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 17, 2025.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
