`NTLoss` - a regression-like loss for LLMs

ntloss is a PyPI package of the "Number Token Loss" for language models. A regression-like loss that improves LLM performance on math tasks. Follows Regress, Don't Guess, ICML 2025

📖 Overview

This repo maintains the code for the ntloss PyPI package

🧑🏽‍💻 Paper source code: Regress, Don't Guess – ICML 2025
📄 Paper: Regress, Don't Guess – A Regression-like Loss on Number Tokens for Language Models
🌐 Project Page: Landing Page
🎮 Demo: HuggingFace Spaces Demo (Streamlit)
📖 Docs: Documentation for the PyPI package

🏃‍♂️ Quick Start

Simply install ntloss into your existing project

uv add ntloss
pip install ntloss # if you are oldschool

Use like this:

from ntloss import NTLoss
ntl_fn = NTLoss(tokenizer=tokenizer)
ntl = ntl_fn(logits, labels)

# We recommend
loss = cross_entropy(logits, labels) + 0.3 * ntl

NOTE: ntloss is currently in alpha phase and pre-release. Feedback & PRs are very welcome.

📝 Citation

If you use ntloss, please cite our paper:

@inproceedings{zausinger2025regress,
  title   = {Regress, Don't Guess – A Regression-like Loss on Number Tokens for Language Models},
  author  = {Jonas Zausinger and Lars Pennig and Anamarija Kozina and Sean Sdahl
             and Julian Sikora and Adrian Dendorfer and Timofey Kuznetsov
             and Mohamad Hagog and Nina Wiedemann and Kacper Chlodny
             and Vincent Limbach and Anna Ketteler and Thorben Prein
             and Vishwa Mohan Singh and Michael Danziger and Jannis Born},
  booktitle = {Proc. of the 42nd International Conference on Machine Learning (ICML)},
  year    = {2025},
  url     = {https://ibm.biz/ntl-main}
}

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Metadata

Release files for ntloss 0.3.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ntloss 0.3.2
File	Size	Uploaded
ntloss-0.3.2.tar.gz	25.2 kB	Aug 13, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for ntloss 0.3.2
File	Interpreter	ABI	Platform	Reset
ntloss-0.3.2-py3-none-any.whl 16.6 kB Aug 13, 2026	Python 3	none	any	Details

Total release size: 41.8 kB

Release files / ntloss-0.3.2.tar.gz

Download URL	ntloss-0.3.2.tar.gz
Size	25.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`01f0fc50cd666d845daa62bbeda669c0959af1f0a9b1054dfc41e229367350fb`
BLAKE2b-256 checksum How to use checksums	`5ad315ac172386c079851bc5258b2db55cb2c7a80234c1dc035af66dccb2ec89`
Upload date	Aug 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.3 {"installer":{"name":"uv","version":"0.12.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / ntloss-0.3.2-py3-none-any.whl

Download URL	ntloss-0.3.2-py3-none-any.whl
Size	16.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`0bb12c0032b9adea0f54cb0cde887459e5ffa9ef907c2138fc1bcbf72b9367e8`
BLAKE2b-256 checksum How to use checksums	`f8167cd6127f45e2094395175e84f651a82ef19c030fcc4bf903072db4f8ce00`
Upload date	Aug 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.3 {"installer":{"name":"uv","version":"0.12.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`