Metadata-Version: 2.4
Name: triton-hf-builder
Version: 0.1.0
Summary: Build Triton model repositories from local Hugging Face checkpoints
License-Expression: Apache-2.0
Project-URL: Repository, https://github.com/ai-lab-tech/triton-hf-builder
Project-URL: Issues, https://github.com/ai-lab-tech/triton-hf-builder/issues
Project-URL: Documentation, https://github.com/ai-lab-tech/triton-hf-builder#readme
Keywords: triton,huggingface,inference,model-repository
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: packaging>=23
Requires-Dist: protobuf<7,>=6.33.5
Requires-Dist: tritonclient[grpc]<3,>=2.73
Provides-Extra: e2e
Requires-Dist: huggingface-hub<1,>=0.30; extra == "e2e"
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: twine<7,>=6; extra == "dev"
Requires-Dist: ruff<1,>=0.9; extra == "dev"
Dynamic: license-file

# triton-hf-builder

[![CI](https://github.com/ai-lab-tech/triton-hf-builder/actions/workflows/tests.yml/badge.svg)](https://github.com/ai-lab-tech/triton-hf-builder/actions/workflows/tests.yml)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](pyproject.toml)
[![Status: Alpha](https://img.shields.io/badge/Status-Alpha-orange.svg)](#adapters-and-compatibility)

**Turn a downloaded Hugging Face checkpoint into a Triton model repository.**

Use the Python library, CLI or the code-server extension in
[Triton Control](https://github.com/ai-lab-tech/triton-control).
Generation reads metadata and copies assets; it does not load weights or require
a GPU. Allow disk space for a full checkpoint copy. **Status: alpha.**

[Python API](docs/python-api.md) · [Serving guide](docs/serving.md) ·
[Serving tests](scripts/README.md) · [Contributing](CONTRIBUTING.md)

## Install

Requires Python 3.10+.

```bash
git clone https://github.com/ai-lab-tech/triton-hf-builder.git
cd triton-hf-builder
python3 -m venv .venv
source .venv/bin/activate
python -m pip install .
```

## Quick start

Point `source` to a complete local checkpoint with weights and tokenizer files.
Choose a new `output` directory outside the checkpoint directory.

```python
from triton_hf_builder import generate_repository

result = generate_repository(
    source="/workspace/hf-checkpoint",
    output="/workspace/model_repository",
    name="my_model",
    runtime="transformers",
    image="nvcr.io/nvidia/tritonserver:26.06-py3",
    device="cpu",
)
print(result.model_directory / "config.pbtxt")
```

**No manual `requirements` needed:** the builder writes runtime defaults into
`requirements.txt`. Optional overrides, all parameters and return values are
explained in the [Python API](docs/python-api.md).

The equivalent CLI:

```bash
triton-hf inspect /workspace/hf-checkpoint
triton-hf build /workspace/hf-checkpoint \
  --output /workspace/model_repository --name my_model \
  --runtime transformers --device cpu \
  --image nvcr.io/nvidia/tritonserver:26.06-py3
```

The output contains the checkpoint assets, `config.pbtxt`, runtime files,
`requirements.txt` and `serving.json`. Deploy this repository to Triton after
preparing its serving dependencies; generation alone does not validate inference.
See the [serving guide](docs/serving.md) for deployment and vLLM path preparation.

## Adapters and compatibility

| Builder adapter (`runtime`) | Triton backend | Model execution | Tasks | Device |
| --- | --- | --- | --- | --- |
| `transformers` | Python | Hugging Face Transformers with PyTorch | Generation, text-to-text, classification, raw features | CPU or GPU |
| `vllm` | vLLM | vLLM engine | Text generation | GPU |
| `tensorrt-llm` | Python | TensorRT-LLM LLM API with PyTorch backend | Text generation | GPU |

The API parameter `runtime` selects the builder adapter. The Triton backend and
model execution framework are separate layers.

Recognized multimodal checkpoints and custom model Python are outside the current
adapter scope. See [runtime limitations](docs/serving.md#runtimes-and-request-contracts).

### Manufacturer compatibility catalog

<!-- compatibility:begin -->

Manufacturer-documented text-generation candidates. [Full catalog and architecture names](compatibility.json).

| Image tag | Engine version | CUDA / reported driver | Architectures |
| --- | --- | --- | --- |
| `26.09-vllm-python-py3` | `0.29.0+3b480f30.nv26.9.68265578` | 13.4.1.012 / 615.71.09 | [118 candidates](https://github.com/vllm-project/vllm/blob/98dff2a81d747d1dba01a47f939f48c3526d4206/docs/models/supported_models.md) † |
| `26.06-vllm-python-py3` | `0.22.1+7b9cb5b7.nv26.6.55098374` | 13.3.0.035 / 610.43.02 | [129 candidates](https://github.com/vllm-project/vllm/blob/0decac0d96c42b49572498019f0a0e3600f50398/docs/models/supported_models.md) † |
| `26.07-trtllm-python-py3` | `1.2.1` | 13.1.0.036 / 590.44.01 | [18 candidates](https://github.com/NVIDIA/TensorRT-LLM/blob/376f7e1bd8ed543f75014309e3fd4b237e9b0e73/docs/source/models/supported-models.md) |
| `26.06-trtllm-python-py3` | `1.2.1` | 13.1.0.036 / 590.44.01 | [18 candidates](https://github.com/NVIDIA/TensorRT-LLM/blob/376f7e1bd8ed543f75014309e3fd4b237e9b0e73/docs/source/models/supported-models.md) |

Source: [NVIDIA image matrix](https://github.com/triton-inference-server/server/blob/eb74dea5d400b92f54eacbd5f3f84903c786e4cd/docs/introduction/compatibility.md). † Upstream base version; NVIDIA build differs.

These are documentation references, not serving guarantees. Check the linked matrices for quantization, GPU and feature restrictions; the reported driver is not a universal minimum.

<!-- compatibility:end -->

The catalog tracks 26.06 and the latest documented image per runtime. A weekly
workflow proposes updates as PRs. [Maintenance and selection policy](scripts/README.md#compatibility-catalog-maintenance).

## Development and releases

See [CONTRIBUTING.md](CONTRIBUTING.md) for local checks and
[RELEASING.md](RELEASING.md) for package publication and code-server integration.
Real GPU serving tests are opt-in; follow the [test guide](scripts/README.md).

[Apache-2.0 license](LICENSE) · [Security policy](SECURITY.md)
Model weights and serving engines retain their own licenses.
