Metadata-Version: 2.4
Name: altamt
Version: 1.1
Summary: ALTAMT: Advanced Lightweight Translation AI Model Transformer - multilingual Kinyarwanda/English/French machine translation with document-level translation, optimized for CPU and GPU inference.
Author: ALTAMT Contributors
License: Apache-2.0
Project-URL: Homepage, https://github.com/altamt-ai/altamt
Project-URL: Documentation, https://github.com/altamt-ai/altamt#readme
Project-URL: Issues, https://github.com/altamt-ai/altamt/issues
Keywords: machine-translation,kinyarwanda,french,multilingual,document-translation,nlp,transformer,low-resource,onnx,quantization
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Requires-Dist: sentencepiece>=0.1.99
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: pyarrow>=14.0
Requires-Dist: sacrebleu>=2.4
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.66
Provides-Extra: onnx
Requires-Dist: onnx>=1.15; extra == "onnx"
Requires-Dist: onnxruntime>=1.17; extra == "onnx"
Provides-Extra: benchmark
Requires-Dist: transformers>=4.40; extra == "benchmark"
Requires-Dist: evaluate>=0.4; extra == "benchmark"
Requires-Dist: psutil>=5.9; extra == "benchmark"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Provides-Extra: all
Requires-Dist: altamt[benchmark,dev,onnx]; extra == "all"
Dynamic: license-file

# altamt

**Multilingual machine translation for Kinyarwanda · English · French — one model, every direction, fast on CPU and GPU.**

`altamt` (Advanced Lightweight Translation AI Model Transformer) is a compact modern Transformer (RMSNorm · SwiGLU · Grouped-Query Attention · RoPE · FlashAttention) that:

- translates **every trained direction with a single model** (`rw↔en`, `rw↔fr`, `en↔fr`, …),
- **auto-detects** the input language when you don't specify it,
- translates **documents of any length** (a paragraph or 20 pages) while preserving paragraphs, headings, lists and line breaks,
- **pivots automatically** through a bridge language for pairs it was never trained on directly,
- runs efficiently on **CPU** (INT8 quantization, KV-cached decoding, physical-core thread tuning) and on **GPU** (bf16/fp16 autocast, token-budget batching).

---

## Contents

1. [Installation](#1-installation)
2. [Quickstart](#2-quickstart)
3. [The `Translator` — full reference](#3-the-translator--full-reference)
4. [Translating: methods and parameters](#4-translating-methods-and-parameters)
5. [Decoding techniques explained](#5-decoding-techniques-explained)
6. [Documents and long text](#6-documents-and-long-text)
7. [Language handling: auto-detection and pivot routing](#7-language-handling-auto-detection-and-pivot-routing)
8. [Performance tuning per device](#8-performance-tuning-per-device)
9. [CLI reference](#9-cli-reference)
10. [Result dictionaries](#10-result-dictionaries)
11. [Troubleshooting](#11-troubleshooting)

---

## 1. Installation

```bash
pip install altamt

# optional extras
pip install "altamt[onnx]"        # ONNX Runtime export & inference
pip install "altamt[benchmark]"   # compare against NLLB / Opus-MT baselines
```

Requires Python ≥ 3.9. CPU-only machines are fully supported — a GPU is never required for inference.

## 2. Quickstart

```python
from altamt import Translator

t = Translator("path/to/checkpoint")          # a trained checkpoint directory

# The target language is required; the source is auto-detected when omitted.
r = t.translate("Mwaramutse nshuti zanjye", tgt_lang="en")
print(r["translated_text"])                   # Good morning my friends
print(r["detected_src"], r["route"])          # rw   rw->en

# Pin the direction explicitly:
t.translate("How are you today?", src_lang="en", tgt_lang="rw")

# Batch — directions can be mixed inside one batch:
t.translate_batch(["Mwaramutse", "Good morning"], tgt_lang=["fr", "rw"])

# A whole document (any length — layout is preserved):
r = t.translate_document(open("rapport.txt").read(), tgt_lang="en")

# A file, in one line (writes rapport.en.txt next to the input):
t.translate_file("rapport.txt", tgt_lang="en")
```

```bash
# Same things from the terminal:
altamt translate "Mwaramutse nshuti zanjye" --model path/to/checkpoint --tgt-lang en
altamt translate-doc rapport.txt --model path/to/checkpoint --tgt-lang en -o rapport.en.txt
```

## 3. The `Translator` — full reference

```python
Translator(
    model_path,                  # checkpoint directory (required)
    tokenizer_path=None,
    device="cpu",
    quantize=False,
    beam_size=4,
    length_penalty=1.0,
    max_new_tokens=512,
    length_ratio=1.8,
    no_repeat_ngram_size=0,
    num_threads="auto",
    batch_tokens=0,
    lid=None,
    pivot_lang="en",
    add_src_lang_tag=None,
    verbose=True,
)
```

| Parameter | Default | What it does |
|---|---|---|
| `model_path` | — | Checkpoint directory produced by training (`best/` or `last/`). Must contain `model.pt`, `config.json`, a SentencePiece `*.model` and — bundled automatically by the trainer — `vocab_config.json`. |
| `tokenizer_path` | `None` | Explicit tokenizer file; by default the first `*.model` inside `model_path` is used. |
| `device` | `"cpu"` | `"cpu"` (the optimized default path), `"cuda"` / `"cuda:N"`, or `"auto"` (CUDA when available, else CPU). Requesting CUDA when it is unavailable raises a clear error containing the exact fix (see `altamt doctor`). |
| `quantize` | `False` | Apply **INT8 dynamic quantization** after loading. CPU only. Typically ~2× lower latency and ~4× smaller weight matrices with negligible quality loss. |
| `beam_size` | `4` | Default beam width for all calls. `1` = greedy (fastest); `4` is the standard quality/speed trade-off (typically +1–2 BLEU over greedy). Overridable per call. |
| `length_penalty` | `1.0` | Beam-search length normalization exponent (Wu et al. 2016). `>1` favors longer outputs, `<1` shorter ones. Only affects beam search. |
| `max_new_tokens` | `512` | Hard **cap** on generated tokens. The *effective* per-batch budget is `min(max_new_tokens, src_tokens × length_ratio + 16)`, so short inputs stay fast and long ones are never truncated mid-sentence. |
| `length_ratio` | `1.8` | Output/input token ratio used for that adaptive budget. Raise it (e.g. `2.5`) if a language pair systematically expands a lot in translation. |
| `no_repeat_ngram_size` | `0` | When `> 0`, any n-gram of this size that already appeared in the output is banned from being generated again. `3` is a good value to stop degenerate repetition loops on noisy or out-of-domain input. `0` = off. |
| `num_threads` | `"auto"` | CPU threading. `"auto"` pins PyTorch to the machine's **physical** cores and disables the inter-op pool (PyTorch's default of one thread per *logical* core over-subscribes SMT machines and makes short-sentence decoding slower). An `int` sets `torch.set_num_threads` exactly; `None` leaves process defaults untouched. |
| `batch_tokens` | `0` | Padded tokens per generation batch for `translate_batch` / documents. `0` selects the per-device default: **4 096 on CPU, 32 768 on CUDA**. The budget is divided by the beam width automatically. |
| `lid` | `None` | A custom `altamt.LanguageDetector`; the built-in rw/en/fr detector is used when omitted. |
| `pivot_lang` | `"en"` | Bridge language for pairs that were not trained directly (see [§7](#7-language-handling-auto-detection-and-pivot-routing)). |
| `add_src_lang_tag` | `None` | Whether to prepend the `<2src>` source-language tag to the encoder input. **Leave it at `None`**: the setting the checkpoint was *trained* with is read from `vocab_config.json`, and a mismatch silently degrades quality with no error. Only override for experiments. |
| `verbose` | `True` | Emit status notices (auto-detected language, pivot routing, document progress) to *stderr* — stdout stays clean for piping. `False` for silent library use. |

Other members: `t.languages` (tuple of supported codes), `t.trained_pairs` (set of directly-trained `(src, tgt)` pairs), `t.warmup()` (one tiny generation to trigger lazy kernel initialization — call it before latency-sensitive serving).

## 4. Translating: methods and parameters

### `translate(text, src_lang=None, tgt_lang=..., beam_size=None, length_penalty=None, max_new_tokens=None)`

Translate a string of **any length**. Short inputs run as a single pass; anything longer than the model's trained sentence length is **transparently segmented, batch-translated and reassembled** (you just get the result — the returned dict then also carries `segments`, `chars_in`, `chars_out`).

- `tgt_lang` — **required**. A multilingual model cannot guess the output language you want.
- `src_lang` — optional; auto-detected when omitted (a stderr notice reports the detection).
- `beam_size`, `length_penalty`, `max_new_tokens` — per-call overrides of the constructor defaults.

### `translate_batch(texts, src_lang=None, tgt_lang=..., beam_size=None, length_penalty=None, max_new_tokens=None)`

Translate many sentences efficiently. Internally the batch is **length-sorted and packed against a token budget**, so almost no compute is spent on padding; results come back in input order.

- `tgt_lang` may be one code for the whole batch **or a list with one code per sentence** — directions mix freely inside one batch, including pivot-routed rows.
- `src_lang=None` auto-detects **per sentence**.

### `translate_document(text, src_lang=None, tgt_lang=..., beam_size=None, length_penalty=None, max_segment_tokens=None, unwrap="auto", progress=False)`

The explicit long-form entry point (what `translate` delegates to for long input), with full control:

- `max_segment_tokens` — tokens per segment; default **80 % of the checkpoint's training `max_src_len`**, which keeps every segment comfortably inside what the model actually saw during training.
- `unwrap` — how multi-line paragraphs are treated:
  - `"auto"` (default): rejoins **hard-wrapped prose** (the shape of text extracted from PDF/DOCX, with a newline at every rendered line) while leaving lists, tables and headings alone;
  - `"never"`: every source line stays its own unit;
  - `"always"`: every multi-line paragraph is joined.
- `progress` — one-line progress notices on stderr.
- Source detection for documents runs over a large sample of the text (not one line), so a stray heading can't misdetect the language.

### `translate_file(input_path, output_path=None, tgt_lang=..., src_lang=None, **document_kwargs)`

Reads a UTF-8 text/Markdown file, translates it as a document, writes the result and returns the written path. Default output: `<stem>.<tgt_lang><suffix>` next to the input (`rapport.txt → rapport.en.txt`).

## 5. Decoding techniques explained

| Technique | Parameter(s) | When to use |
|---|---|---|
| **Greedy decoding** | `beam_size=1` | Fastest; ~1–2 BLEU below beam 4. Right for interactive/low-latency use and quick experiments. |
| **Beam search** | `beam_size=4` (default), higher for maximum quality | Explores several hypotheses in parallel and keeps the best-scoring finished one. Cost grows with beam width; beyond ~8 the quality gains vanish. |
| **Length normalization** | `length_penalty` | Beam search naturally favors short outputs (fewer log-prob terms). The Wu et al. normalization divides scores by `((5+len)/6)^penalty`. `1.0` is neutral-ish; raise toward `1.2–1.4` if outputs feel clipped, lower toward `0.8` if they ramble. |
| **Adaptive generation budget** | `max_new_tokens` + `length_ratio` | The per-batch budget is `min(max_new_tokens, src_tokens × length_ratio + 16)`. This is why long sentences don't get cut off mid-way and short ones don't waste decode steps. |
| **Repetition ban** | `no_repeat_ngram_size` | Bans regenerating any already-emitted n-gram of that size. `3` cures the classic "the the the…" / phrase-loop failure on noisy input at negligible cost. |
| **Minimum length** | (internal, documents) | Document segments decode with a 1-token minimum so a segment can never come back empty. |
| **INT8 quantization** | `quantize=True` | CPU-only. Dynamic per-batch quantization of all Linear layers; ~2× speed, ~4× smaller matmuls, negligible quality change. |
| **KV-cached decoding** | always on | Decoder self-attention states are cached per step and encoder cross-attention keys/values are computed once, so per-token cost is independent of how much has been generated. |
| **FlashAttention + GQA** | always on (PyTorch ≥ 2.5, supported hardware) | Causality is declared (`is_causal=True`) rather than materialized as a mask, which makes the flash kernel eligible; grouped KV heads are consumed natively (`enable_gqa`). Zero configuration. |

## 6. Documents and long text

A sentence-level model handed a five-page document as one sequence is far outside its training distribution — that's why naive tools translate a paragraph well and garble a report. `altamt` instead:

1. **Splits on blank lines**, keeping the exact separators, so paragraph spacing survives byte-for-byte.
2. **Unwraps hard-wrapped prose** (PDF/DOCX extraction artifacts) into logical paragraphs; keeps headings, bullets, numbered items and table rows as separate units.
3. **Strips leading markers** (`- `, `1. `, `## `, `(a) `) before translation and re-attaches them after — the model translates prose, not punctuation scaffolding. Titles and initials (`M. Dupont`, `J. R. R. Tolkien`) are *not* mistaken for list markers.
4. **Splits sentences** with an abbreviation-, initial- and decimal-aware splitter tuned for English, French and Kinyarwanda (`fig. 3`, `3.5`, `Dr. Smith` don't split).
5. **Packs sentences** into segments up to the token budget, so short sentences keep their mutual context and nothing exceeds the trained length. A single over-long sentence splits at clause boundaries first.
6. **Passes through** rules, bare URLs and numeric-only rows verbatim instead of hallucinating over them.
7. **Batches everything**: the whole document is translated in length-sorted token-budgeted batches — a 20-page document is one efficient pass, not thousands of single-sentence calls.

With an identity translation, reassembly is byte-identical to the input.

```python
r = t.translate_document(text, tgt_lang="en", progress=True)
r["segments"], r["chars_in"], r["chars_out"]     # bookkeeping
```

## 7. Language handling: auto-detection and pivot routing

**Auto-detection.** When `src_lang` is omitted, a dependency-free character-trigram Naive-Bayes detector (with a function-word prior; microsecond latency) picks the source among the model's languages, excluding the requested target. It ships with rw/en/fr profiles and can be retrained on your own corpora:

```python
from altamt import LanguageDetector
det = LanguageDetector.train({"rw": ["mono.rw.txt"], "sw": ["mono.sw.txt"]})
det.save("lid.json")
t = Translator("ckpt", lid=LanguageDetector.load("lid.json"))
```

**Pivot routing.** Every checkpoint records which pairs it was actually trained on (`trained_pairs` in `vocab_config.json`). A directly-trained pair decodes in one hop. An untrained pair is bridged automatically through `pivot_lang` (default English) — e.g. `rw→fr` as `rw→en→fr` — and the result reports the full `route`, the `pivot` language and the intermediate `pivot_text`. If no route exists, a `ValueError` lists the trained pairs. Models trained on direct data for all pairs (the recommended setup) never pivot.

## 8. Performance tuning per device

### CPU

```python
t = Translator("ckpt", quantize=True, num_threads="auto", beam_size=1)
t.warmup()
```

- `num_threads="auto"` — pins to physical cores; the single most common CPU misconfiguration is letting PyTorch use every SMT thread.
- `quantize=True` — INT8; ~2× faster.
- `beam_size=1` for latency-critical paths, `4` for quality.
- Batching helps a lot even on CPU: prefer one `translate_batch` / `translate_document` call over a Python loop of `translate` calls.

### GPU

```python
t = Translator("ckpt", device="cuda")          # or device="auto"
```

- Generation runs under **bf16/fp16 autocast** automatically.
- The generation batch budget defaults to 32 768 padded tokens — documents and large batches saturate the GPU without further tuning; set `batch_tokens` to trade memory for throughput.
- INT8 dynamic quantization is CPU-only; don't combine it with CUDA.

### Serving tips

- Construct the `Translator` **once** and reuse it; construction loads weights and the tokenizer.
- Call `warmup()` once at startup.
- `verbose=False` in servers; all notices go to stderr otherwise.

## 9. CLI reference

Every command: `altamt <command> --help` shows all flags.

### `altamt translate` — text in, translation out

```bash
altamt translate "Mwaramutse" --model CKPT --tgt-lang en
echo "Mwaramutse" | altamt translate --model CKPT --tgt-lang en
```

| Flag | Default | Meaning |
|---|---|---|
| `--tgt-lang` | required | Target language code. |
| `--src-lang` | auto | Source language (auto-detected when omitted). |
| `--beam-size` | 4 | Beam width (1 = greedy). |
| `--max-new-tokens` | 512 | Generation cap (adaptive budget below it). |
| `--int8` | off | INT8 dynamic quantization (CPU). |
| `--device` | cpu | `cpu` / `cuda` / `cuda:N` / `auto`. |
| `--threads` | auto | `auto`, an integer, or `default`. |
| `--json` | off | Print the full result dict instead of just the text. |

Long input is chunked transparently — the same command works for a sentence or a whole pasted article.

### `altamt translate-doc` — files and multi-page documents

```bash
altamt translate-doc rapport.txt --model CKPT --tgt-lang en -o rapport.en.txt
cat report.md | altamt translate-doc --model CKPT --tgt-lang rw > report.rw.md
```

Adds to the flags above: `--output/-o` (default `<stem>.<tgt><suffix>`; stdout when reading stdin), `--segment-tokens` (default 80 % of the model's trained length), `--unwrap {auto,always,never}`, `--no-repeat-ngram N`, `--quiet`.

### `altamt benchmark` — reportable numbers

```bash
altamt benchmark --model CKPT --test-file test.json --beam-size 4 \
    --device cpu --int8 --output-dir results --formats md latex
```

Reports per direction and per system: **SacreBLEU** (with its citable signature), **chrF++**, latency (ms/sentence), output tokens/sec, peak RAM and parameter count — as `report.md` and a booktabs `report.tex`. `--baseline facebook/nllb-200-distilled-600M` (repeatable) runs Hugging Face baselines through the *same* harness (`pip install altamt[benchmark]`). Run once with `--device cpu` and once with `--device cuda` for both columns.

### `altamt train` / `altamt train-tokenizer` — training

```bash
altamt train-tokenizer --files data.json --output tokenizer/ --languages rw en fr
altamt train --config configs/base.yaml
torchrun --nproc_per_node=8 -m altamt.cli train --config configs/base.yaml
```

**Every configuration value doubles as a CLI flag**: `--training.lr 1e-4`, `--model.encoder_layers 16`, `--data.languages rw,en,fr`, or the `--set section.field=value` form. CLI wins over the YAML/JSON file; unset values keep the file's (or built-in) defaults; the effective config is saved into the run directory. See the [GitHub README](https://github.com/altamt-ai/altamt#readme) for the full training guide (token-budget batching, DDP/DeepSpeed, fine-tuning, resuming).

### `altamt quantize` / `altamt export-onnx` — compression & export

```bash
altamt quantize    --model CKPT --output-dir int8_model
altamt export-onnx --model CKPT --output-dir onnx_model
```

The PyTorch INT8 path keeps the KV cache (recommended low-latency route); ONNX export produces portable `encoder.onnx` / `decoder.onnx` graphs.

### `altamt doctor` — GPU/PyTorch environment diagnosis

```bash
altamt doctor          # diagnose: wheel CUDA vs driver CUDA, exact fix commands
altamt doctor --fix    # reinstall the torch wheel matching your driver
```

## 10. Result dictionaries

`translate` / each element of `translate_batch`:

| Key | Type | Meaning |
|---|---|---|
| `translated_text` | str | The translation. |
| `detected_src` | str | Resolved source language. |
| `src_autodetected` | bool | Whether the source was auto-detected. |
| `tgt` | str | Target language. |
| `route` | str | `"rw->en"`, or `"rw->en->fr"` when pivoted. |
| `pivot` | str \| None | Bridge language, if pivot-routed. |
| `pivot_text` | str \| None | Intermediate translation, if pivot-routed. |
| `latency_ms` | float | Wall time (per sentence for batches). |

`translate_document` / long inputs to `translate` add: `segments` (int), `chars_in` / `chars_out` (int).

## 11. Troubleshooting

| Symptom | Cause & fix |
|---|---|
| `tgt_lang is required` | Always pass the target language — a multilingual model can't guess it. |
| `No translation route from X to Y` | The pair isn't trained and no pivot bridges it; the error lists trained pairs. Train with direct data or add a pivot leg. |
| Output repeats a phrase forever | Set `no_repeat_ngram_size=3`. |
| Long text came out truncated/garbled on an old version | Upgrade — long input is now segmented automatically (`translate` and `translate-doc`). |
| `device='cuda' requested but CUDA is not available` | The error includes the exact fix; or run `altamt doctor --fix`. |
| Slow on CPU | `quantize=True`, `num_threads="auto"`, `beam_size=1`, batch your inputs, call `warmup()`. |
| Wrong auto-detected language on very short text | Pass `src_lang` explicitly — one or two words are genuinely ambiguous. |

## License

Apache 2.0 — full architecture, data-format, training and benchmarking documentation in the [GitHub README](https://github.com/altamt-ai/altamt#readme).
