HL Gauss - Pytorch
The Gaussian Histogram Loss (HL-Gauss) proposed by Imani et al. with a few convenient wrappers, in Pytorch.
A team at Deepmind wrote a paper with a lot of positive findings for its use in value-based RL.
Put into action here, seems to work well
Install
$ pip install hl-gauss-pytorch
Usage
The HLGaussLoss module as defined in Appendix A. of the Stop Regressing paper
import torch
from hl_gauss_pytorch import HLGaussLoss
hl_gauss = HLGaussLoss(
min_value = 0.,
max_value = 5.,
num_bins = 32,
sigma = 0.5,
clamp_to_range = True # this was added because if any values fall outside of the bins, the loss is 0 with the current logic
)
logits = torch.randn(3, 16, 32).requires_grad_()
targets = torch.randint(0, 5, (3, 16)).float()
loss = hl_gauss(logits, targets)
loss.backward()
# after much training
pred_target = hl_gauss(logits) # (3, 16)
For a convenient layer that predicts from embedding to logits, import HLGaussLayer
import torch
from hl_gauss_pytorch import HLGaussLayer
hl_gauss_layer = HLGaussLayer(
dim = 256, # input embedding dimension
hl_gauss_loss = dict(
min_value = 0.,
max_value = 5.,
num_bins = 32,
sigma = 0.5,
)
)
embed = torch.randn(7, 256)
targets = torch.randint(0, 5, (7,)).float()
loss = hl_gauss_layer(embed, targets)
loss.backward()
# after much training
pred_target = hl_gauss_layer(embed) # (7,)
For ablating the proposal, you can make the HLGaussLayer fall back to regular regression by setting use_regression = True, keeping the code above unchanged
HLGaussLayer(..., use_regression = True)
Citations
@article{Imani2024InvestigatingTH,
title = {Investigating the Histogram Loss in Regression},
author = {Ehsan Imani and Kai Luedemann and Sam Scholnick-Hughes and Esraa Elelimy and Martha White},
journal = {ArXiv},
year = {2024},
volume = {abs/2402.13425},
url = {https://api.semanticscholar.org/CorpusID:267770096}
}
@inproceedings{Imani2018ImprovingRP,
title = {Improving Regression Performance with Distributional Losses},
author = {Ehsan Imani and Martha White},
booktitle = {International Conference on Machine Learning},
year = {2018},
url = {https://api.semanticscholar.org/CorpusID:48365278}
}
@article{Farebrother2024StopRT,
title = {Stop Regressing: Training Value Functions via Classification for Scalable Deep RL},
author = {Jesse Farebrother and Jordi Orbay and Quan Ho Vuong and Adrien Ali Taiga and Yevgen Chebotar and Ted Xiao and Alex Irpan and Sergey Levine and Pablo Samuel Castro and Aleksandra Faust and Aviral Kumar and Rishabh Agarwal},
journal = {ArXiv},
year = {2024},
volume = {abs/2403.03950},
url = {https://api.semanticscholar.org/CorpusID:268253088}
}
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file hl_gauss_pytorch-0.2.4.tar.gz.
File metadata
- Download URL: hl_gauss_pytorch-0.2.4.tar.gz
- Upload date:
- Size: 8.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.8.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7807b1e22694ab539e8096373a6d906530940dcd103dd2c238385683e1517d42
|
|
| MD5 |
e9af2e1eb041a8a5c66959549314613f
|
|
| BLAKE2b-256 |
1a43339a2cc8bcd817edf3ed6f80a7808d2b5838f5bc6fb51ec538a1ef112a7a
|
File details
Details for the file hl_gauss_pytorch-0.2.4-py3-none-any.whl.
File metadata
- Download URL: hl_gauss_pytorch-0.2.4-py3-none-any.whl
- Upload date:
- Size: 9.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.8.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
680dc51ddf30a424b4c4480d5d178d6c4d63ff7904866db0fe1c3b0275f49e19
|
|
| MD5 |
aa0b214379bbe0a5ef61a2a956551ae6
|
|
| BLAKE2b-256 |
e0cb4cf94d75c18192f927759339bbbd8190e6138e6b129abf47ee74dc555574
|
