Metadata-Version: 2.4
Name: ntloss
Version: 0.3.1
Summary: Number Token Loss - A regression-alike loss to improve numerical reasoning in language models
Author-email: Jannis Born <jab@zurich.ibm.com>, Lars Pennig <lpennig@t-online.de>, Jonas Zausinger <jonas.zausinger@tum.de>, Sarah de Ruiter <Sarah.Louise.De.Ruiter@ibm.com>, Roland Giovannangeli <roland.giovannangeli@gmail.com>
License: MIT
Project-URL: Source code, https://github.com/AI4SD/number-token-loss
Project-URL: Bug Tracker, https://github.com/AI4SD/number-token-loss/issues
Project-URL: Paper, https://arxiv.org/abs/2411.02083
Project-URL: Demo, https://huggingface.co/spaces/jannisborn/NumberTokenLoss
Project-URL: Landing Page, https://tum-ai.github.io/number-token-loss/
Keywords: Machine Learning,Language Models,Number Token Loss,AI4Math,LLM,LLM Training,Reasoning models
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: transformers>=4.55.0
Requires-Dist: loguru>=0.7.3
Requires-Dist: torch>=2.7.1
Dynamic: license-file

<div align="center">


# `NTLoss` - a regression-like loss for LLMs


[![Paper](https://img.shields.io/badge/Paper-ICML-darkgreen.svg)](https://ibm.biz/ntl-paper)
[![Landing](https://img.shields.io/badge/Landing-Page-blue.svg)](https://ibm.biz/ntl-main)
[![Demo](https://img.shields.io/badge/🤗-Demo-yellow.svg)](https://ibm.biz/ntl-demo)
[![CI](https://github.com/AI4SD/number-token-loss/actions/workflows/ci.yaml/badge.svg)](https://github.com/AI4SD/number-token-loss/actions/workflows/ci.yaml)
[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![PyPI](https://img.shields.io/pypi/v/ntloss?label=pypi&color=brightgreen)](https://pypi.org/project/ntloss/)
[![Docs](https://github.com/AI4SD/number-token-loss/actions/workflows/docs.yaml/badge.svg)](https://ibm.biz/ntl-docs)
[![Downloads](https://static.pepy.tech/badge/ntloss)](https://pepy.tech/project/ntloss)

*`ntloss` is a PyPI package of the "Number Token Loss" for language models. A regression-like loss that improves LLM performance on math tasks. Follows* **Regress, Don't Guess, ICML 2025**


</div>

---

## 📖 Overview
This repo maintains the code for the `ntloss` [PyPI package](https://pypi.org/project/ntloss/)

- 🧑🏽‍💻 **Paper source code**: [Regress, Don't Guess – ICML 2025](https://ibm.biz/ntl-code)
- 📄 **Paper**: [Regress, Don't Guess – A Regression-like Loss on Number Tokens for Language Models](https://ibm.biz/ntl-paper)
- 🌐 **Project Page**: [Landing Page](https://ibm.biz/ntl-main)
- 🎮 **Demo**: [HuggingFace Spaces Demo (Streamlit)](https://ibm.biz/ntl-demo)
- 📖 **Docs**: [Documentation for the PyPI package](https://ibm.biz/ntl-docs)


## 🏃‍♂️ Quick Start


Simply install `ntloss` into your existing project
```sh
uv add ntloss
pip install ntloss # if you are oldschool
```

Use like this:
```py
from ntloss import NTLoss
ntl_fn = NTLoss(tokenizer=tokenizer)
ntl = ntl_fn(logits, labels)

# We recommend
loss = cross_entropy(logits, labels) + 0.3 * ntl
```

NOTE: `ntloss` is currently in alpha phase and pre-release. Feedback & PRs are very welcome.


## 📝 Citation

If you use `ntloss`, please cite our paper:

```bibtex
@inproceedings{zausinger2025regress,
  title   = {Regress, Don't Guess – A Regression-like Loss on Number Tokens for Language Models},
  author  = {Jonas Zausinger and Lars Pennig and Anamarija Kozina and Sean Sdahl
             and Julian Sikora and Adrian Dendorfer and Timofey Kuznetsov
             and Mohamad Hagog and Nina Wiedemann and Kacper Chlodny
             and Vincent Limbach and Anna Ketteler and Thorben Prein
             and Vishwa Mohan Singh and Michael Danziger and Jannis Born},
  booktitle = {Proc. of the 42nd International Conference on Machine Learning (ICML)},
  year    = {2025},
  url     = {https://ibm.biz/ntl-main}
}
```

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
