Metadata-Version: 2.4
Name: anthracite
Version: 2.0.3
Summary: Production-oriented PyTorch training and inference framework with modern Anthracite decoder architecture, distributed training, external tokenizer support, and efficient KV caching.
License: MIT
Keywords: ai,deep-learning,llm,training,fine-tuning,tokenizer,nlp
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0
Requires-Dist: numpy>=1.24
Requires-Dist: safetensors>=0.4
Requires-Dist: tqdm>=4.65
Requires-Dist: transformers<6,>=4.36
Requires-Dist: tokenizers<1,>=0.15
Requires-Dist: sentencepiece>=0.2
Requires-Dist: tiktoken>=0.7
Requires-Dist: huggingface_hub>=0.20
Provides-Extra: hf
Requires-Dist: datasets>=2.14; extra == "hf"
Requires-Dist: huggingface_hub>=0.20; extra == "hf"
Requires-Dist: tokenizers>=0.15; extra == "hf"
Requires-Dist: transformers>=4.36; extra == "hf"
Provides-Extra: tpu
Requires-Dist: torch_xla>=2.1; extra == "tpu"
Provides-Extra: all
Requires-Dist: datasets>=2.14; extra == "all"
Requires-Dist: huggingface_hub>=0.20; extra == "all"
Requires-Dist: tokenizers>=0.15; extra == "all"
Requires-Dist: transformers>=4.36; extra == "all"
Requires-Dist: torch_xla>=2.1; extra == "all"
Dynamic: license-file

# Anthracite 2.0.3

Anthracite is a PyTorch-first training and inference framework for building text-generation and embedding models from scratch or fine-tuning existing Anthracite checkpoints.

Version 2.0.3 focuses on predictable distributed startup, external-tokenizer compatibility, efficient decoder inference, safer checkpoints, and a cleaner production workflow.

## What 2.0.3 changes

### 1. Multi-GPU external tokenizer fix

The old failure mode was especially confusing because the real tokenizer backend error occurred inside a spawned worker and PyTorch reported it as `ProcessRaisedException` plus a `SIGTERM` for another worker.

2.0.3 changes the startup order:

1. The parent process loads the supplied tokenizer first.
2. The parent materializes a self-contained tokenizer bundle in `.anthracite-cache/tokenizer`.
3. Anthracite round-trip loads that exact bundle before GPU workers start.
4. Workers receive the shared tokenizer path instead of reconstructing the original live tokenizer object.
5. A failed tokenizer backend is therefore a normal preflight error, not a secondary multiprocessing traceback.

Supported external tokenizer forms include Hugging Face tokenizer directories, `tokenizer.json` bundles, Hugging Face repo identifiers, low-level `tokenizers` objects, and live custom objects with `encode`/`decode` methods. For automatic multi-GPU worker startup, the supplied tokenizer must also be persistable through `save_pretrained()` or `save(path)`.

For slow tokenizer families, Anthracite tries the Transformers slow tokenizer path before the fast conversion path. The distribution also includes `sentencepiece` and `tiktoken` so common slow-tokenizer backends are available after installation.

### 2. Anthracite-2.0 decoder architecture

The default text architecture is `anthracite-2.0` and keeps the framework's own decoder design while improving the implementation:

- RMSNorm pre-normalization, using PyTorch's fused `rms_norm` when available.
- Rotary position encoding (RoPE).
- Grouped-query attention (GQA) with separate Q and KV head counts.
- PyTorch scaled-dot-product attention (SDPA) when available, with a math fallback.
- Q/K normalization can be enabled with `qk_norm=True`.
- SwiGLU feed-forward blocks.
- Weight-tied input/output embeddings by default.
- Optional linear or NTK-style RoPE scaling controls.
- Gradient checkpointing support for memory-constrained training.
- Preallocated per-layer KV cache for low-allocation autoregressive generation.

The KV cache is not a claim that generation is magically constant-time: each new token still attends to the existing context. The important improvement is that cached K/V tensors are appended in-place rather than repeatedly concatenated into new tensors.

### 3. Safer checkpoints and resume

Checkpoints are written into a temporary directory and become visible only after all required files finish writing. This reduces the chance of a partially written checkpoint being treated as the latest good checkpoint.

Resume is strict by default. If the saved tensors do not match the current architecture, Anthracite raises a checkpoint error instead of silently ignoring missing or unexpected tensors. Set `strict_resume=False` only for an intentional migration.

## Installation

Anthracite 2.0 targets modern Python and PyTorch environments.

```bash
pip install .
```

The package declares the tokenizer backends commonly required by external Hugging Face tokenizers (`tokenizers`, `sentencepiece`, and `tiktoken`).

For TPU support:

```bash
pip install .[tpu]
```

For Hugging Face datasets:

```bash
pip install .[hf]
```

## Quick start

```python
from anthracite import train, generate

train(
    model_name="Saraswati-3M",
    model_type="text_gen",
    dataset="data.txt",
    tokens="100M",
    params="3M",
    context_length=512,
    tokenizer="./tokenizer/",
    device="auto",
    precision="bf16",
)

print(generate("./models/Saraswati-3M", "Hello,", max_new_tokens=80))
```

When `device="auto"` detects more than one CUDA GPU on a single machine, Anthracite can launch one distributed worker per GPU. The distributed path uses DDP/data parallelism; the model replica must therefore fit on each GPU.

## External tokenizer example

```python
from transformers import AutoTokenizer
from anthracite import train

tok = AutoTokenizer.from_pretrained("gpt2", use_fast=False)

train(
    model_name="ExternalTokModel",
    dataset="data.txt",
    tokens="50M",
    params="20M",
    tokenizer=tok,
    device="auto",
)
```

A local tokenizer bundle also works:

```python
train(
    model_name="LocalTokModel",
    dataset="data.txt",
    tokens="50M",
    params="20M",
    tokenizer="./my-tokenizer/",
)
```

For a multi-GPU run, the parent process persists and validates the tokenizer before workers start. The original tokenizer object is not expected to be reconstructed independently in each worker.

## Using Anthracite-2.0 directly

The public training API forwards every `TrainingConfig` field through `**kwargs`, including the new attention/cache controls:

```python
train(
    model_name="ModernModel",
    dataset="data.txt",
    params="100M",
    context_length=2048,
    attention_backend="auto",   # auto | sdpa | math
    rope_scaling="ntk",         # none | linear | ntk
    use_cache=True,
    grad_checkpointing=True,
)
```

`attention_backend="auto"` prefers PyTorch SDPA. `attention_backend="math"` is the explicit implementation fallback. `grad_checkpointing=True` trades extra recomputation for lower activation memory and is intended for training, not cached generation.

## Multi-GPU execution

Automatic single-node launch:

```python
train(
    model_name="MultiGPUModel",
    dataset="data.txt",
    tokens="1B",
    params="300M",
    device="multi-gpu",
)
```

Or launch your script with PyTorch's distributed launcher:

```bash
torchrun --standalone --nproc-per-node=4 train.py
```

Anthracite does not claim that DDP can train a model larger than a single GPU's VRAM. DDP replicates the model on every rank. For model sizes that cannot fit on one GPU, use a sharded strategy such as PyTorch FSDP2 rather than assuming DDP will solve the memory problem.

## KV-cache generation

Normal generation automatically uses Anthracite-2.0's KV cache:

```python
from anthracite import generate

text = generate(
    "./models/ModernModel",
    "Once upon a time",
    max_new_tokens=256,
    temperature=0.8,
    top_p=0.95,
)
```

Streaming generation also reuses the incremental cache rather than regenerating the entire prefix for every emitted token:

```python
from anthracite import stream_text

for piece in stream_text("./models/ModernModel", "Once upon a time", max_tokens=256):
    print(piece, end="", flush=True)
```

## Checkpoints and resume

```python
train(
    model_name="ResumeMe",
    dataset="data.txt",
    tokens="500M",
    params="100M",
    checkpoint_interval=2000,
    resume=True,
)
```

To intentionally load a compatible-but-not-identical checkpoint during migration:

```python
train(
    model_name="Migrated",
    dataset="data.txt",
    tokens="10M",
    params="100M",
    resume="./models/old/checkpoints/checkpoint-5000",
    strict_resume=False,
)
```

Use `strict_resume=False` deliberately. It can hide an architectural mismatch and should not be the default for ordinary resume.

## SFT and embeddings

SFT remains part of the training pipeline and can automatically detect common instruction/chat record layouts. Input masking can be enabled with `sft_mask_input=True` so only answer tokens contribute to the supervised loss.

Embedding models are supported through the existing `model_type="embedding"` pipeline with Anthracite or BERT-style backbones and mean/CLS/max pooling.

## Model size and hardware reality

`params="..."` is a target budget. Anthracite solves that target into concrete width/depth/head dimensions; the final exact parameter count is stored in the model config and metadata.

A model's usefulness is not determined only by parameter count. Training quality also depends on tokenizer quality, data quality, token budget, optimization settings, context length, and hardware. Anthracite does not promise that a small training run will produce a competitive general-purpose assistant.

Anthracite can build substantially larger models when the architecture dimensions and hardware support them, but this 2.0.3 release's built-in distributed training path is DDP-style replication rather than full model sharding.

## Output layout

A successful model directory contains the model weights, model config, tokenizer artifacts, training config, metrics, metadata, and a generated model card. Intermediate distributed artifacts live under `.anthracite-cache` and include the validated shared tokenizer and, when enabled, a reusable token cache.

## Validation done for this 2.0.3 drop

The repository includes a small regression suite covering:

- external `tokenizer.json` round-tripping without the Rust tokenizer package present;
- persisted custom tokenizer-object round-tripping;
- Anthracite-2.0 forward/loss smoke tests;
- incremental KV-cache decode;
- cache-backed generation beyond the base context window;
- SDPA/math backend shape checks;
- version/config defaults;
- strict syntax compilation across the package.

Actual NCCL multi-GPU and TPU hardware execution is environment-dependent. Those paths still need to be exercised on the target hardware before a production training job.

## Architecture compatibility

Existing legacy architecture names remain accepted for compatibility:

- `anthracite-1`
- `anthracite-1.0`
- `anthracite-1.2`
- `gpt2`
- `bert` (embedding pipeline)

New text models default to `anthracite-2.0`.

## License

MIT. See `LICENSE`.


## 2.0.3 tokenizer reliability

External Hugging Face tokenizer bundles are materialized and validated in a parent-process staging directory before distributed workers start. Persisted special-token IDs are authoritative, including GPT-2's shared BOS/EOS id 50256.
