LocalSoftmax
Local Softmax parallelize the softmax computation by splitting the tensor into smaller sub-tensors and applying the softmax function on each of these smaller tensors independently. In other words, we want to compute a "local" softmax on each chunk of the tensor, instead of on the entire tensor.
Appreciation
- Lucidrains
- Agorians
Install
pip install local-sftmx
Usage
import torch
from local_sfmx import local_softmax
tensor = torch.rand(10, 5)
result = local_softmax(tensor, 2)
print(result)
Algorithm
function LocalSoftmax(tensor, num_chunks):
split tensors into num_chunks smaller tensors
for each smaller tensor:
apply standard softmax
concatenate the results
return concatenated tensor
License
MIT
1 maintainer
Release files for local-sfmx 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| local_sfmx-0.0.4.tar.gz | 4.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| local_sfmx-0.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 9.3 kB
Release files / local_sfmx-0.0.4.tar.gz
| Download URL | local_sfmx-0.0.4.tar.gz |
|---|---|
| Size | 4.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e624b37aabce4a29d453dc113c41b7af09c0910b9a6f00bba1417de4905d2c09
|
|
BLAKE2b-256 checksum How to use checksums |
791140c77140d775f750eeaa0c66b2fa980c1c8e66c3cbd5eb7c1b3fa277c66d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Release files / local_sfmx-0.0.4-py3-none-any.whl
| Download URL | local_sfmx-0.0.4-py3-none-any.whl |
|---|---|
| Size | 4.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f951424a700eedaf780c2fd730ad71a0f4d1238aa0268600a7f70f4299a4370f
|
|
BLAKE2b-256 checksum How to use checksums |
ab64ea6bac3a2204ded329774b333720db8b69f05ba3a7323dcfa267e0d58b1e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
