A fast, yet specialized, RMSNorm/LayerNorm implementation
Project description
# faster-norm
A fast, yet specialized, RMSNorm/LayerNorm implementation
This library is under development. Currently, only some special cases are supported, and the performance is not yet fully optimized.
[x] RMSNorm
[ ] LayerNorm
[x] Float16 and BFloat16
[ ] More data types
[x] More shapes
[ ] Accelerate if no wgrad
[ ] Performance tuning
## Statement
This work was independently completed by me at home using my personal RTX 3080.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
faster_norm-0.2.1.tar.gz
(6.1 kB
view details)
File details
Details for the file faster_norm-0.2.1.tar.gz.
File metadata
- Download URL: faster_norm-0.2.1.tar.gz
- Upload date:
- Size: 6.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.11.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 44a5b85ac423656f2b398425f1212e43e1a1e2e66ab9d5a29fda48e801175ddf |
|
| MD5 | 60c1e9a6fe3b0afdf5e2946541a7a25b |
|
| BLAKE2b-256 | 9b7f9d9e325ab19b615421b7d6abf294784e2776102eeec83e57ea98f0f8573f |
