Metadata-Version: 2.4
Name: mammoth-nlp
Version: 0.3.0rc1
Summary: Massively Multilingual Modular Open Translation @ Helsinki
Project-URL: Documentation, https://helsinki-nlp.github.io/mammoth/
Project-URL: Source, https://github.com/Helsinki-NLP/mammoth
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: torch>=2.8
Requires-Dist: numpy>=2.0
Requires-Dist: einops>=0.8.1
Requires-Dist: transformers>=4.57
Requires-Dist: tokenizers>=0.22
Requires-Dist: huggingface-hub>=0.34.4
Requires-Dist: safetensors>=0.6.2
Requires-Dist: PyYAML>=6.0.2
Requires-Dist: ConfigArgParse>=1.7.1
Requires-Dist: click>=8.0
Requires-Dist: tqdm>=4.67.1
Requires-Dist: tensorboard>=2.20.0
Requires-Dist: sacrebleu>=2.5.1
Provides-Extra: sentencepiece
Requires-Dist: sentencepiece==0.2.1; extra == "sentencepiece"
Provides-Extra: affinity
Requires-Dist: psutil; extra == "affinity"
Dynamic: description
Dynamic: description-content-type
Dynamic: license-file
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# 🦣 MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki

This repository contains the code for 🦣 MAMMOTH, the modular translation toolkit from Helsinki-NLP.

This library is built on top of OpenNMT-py.
[OpenNMT-py](https://github.com/OpenNMT/OpenNMT-py) is the [PyTorch](https://github.com/pytorch/pytorch) version of the [OpenNMT](https://opennmt.net) project, an open-source (MIT) neural machine translation framework. It is designed to be research friendly to try out new ideas in translation, summary, morphology, and many other domains. Some companies have proven the code to be production ready.

## Getting started

```bash
pip install mammoth-nlp
```

- [Installation guide](docs/source/install.md): install locally or on a specific cluster.
- [Quickstart](docs/source/quickstart.md): run a first training with synthetic toy data and a small translation model.
- [Documentation site](https://helsinki-nlp.github.io/mammoth/): the full Mammoth documentation (currently under development).
  The original ONMT-py documentation is available [here](https://opennmt.net/OpenNMT-py/).

## Guides
- [LUMI/Roihu quickstart guide](docs/source/CSC_quickstart.md): working on the LUMI/Roihu supercomputer environments.
- [Exporting Mammoth to Model Hub guide](docs/source/exporting_to_huggingface.md): export pretrained Mammoth model weights to the Hugging Face Model Hub.
- [HF Tokenizers guide](docs/source/HF_TOKENIZERS.md): we recommend the Hugging Face Tokenizer for training and inferencing in Mammoth.
- [Indexed dataset guide](docs/source/INDEXED_DATASET_GUIDE.md): tokenize once up front to remove tokenization cost during training.

**Note:** We would greatly appreciate issue reports for this repository and its documentation.

## Acknowledgements

We thank the NVIDIA AI Technology Center Finland for their help with the multi-gpu/node implementation.
