Metadata-Version: 2.4
Name: tml-renderers
Version: 0.2.0
Classifier: Typing :: Typed
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: Apache Software License
License-File: LICENSE
License-File: NOTICE
License-File: THIRD_PARTY_LICENSES.md
Summary: Public TML chat types, tokenizers, and native renderers
Keywords: chat,message,tml
Author-email: Thinking Machines Lab <tinker@thinkingmachines.ai>
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# tml-renderers

`tml-renderers` turns structured text, image, and audio chat into model-ready
tokens and media for training and inference. It supports Thinking Machines Lab
models, including Inkling and Inkling-Small, through TMLv0, and the
[supported third-party models served on Tinker](https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/)
through model-specific renderers.

## Installation

Always use the latest release:

```sh
pip install --upgrade tml-renderers
```

`tml-renderers` is a compiled extension that requires PyTorch 2.10 or newer.

## Quickstart

Create a short conversation with native chat objects and render it for
inference or training:

```py
from tml_renderers import chat, renderers

user = chat.Message(
    content=chat.Text("Hello, world!"),
    author=chat.Author(chat.AuthorKind.User),
)
assistant = chat.Message(
    content=chat.Text("Hello!"),
    author=chat.Author(chat.AuthorKind.Model),
)
native_messages = chat.MessageList([user, assistant])

renderer = renderers.Qwen3()
spans, parser = renderer.render_for_completion([user])
training_examples = renderer.render_for_sft(native_messages)
```

To continue a conversation, add the model response and new user or tool
messages to the history and render again.

## Supported models

| Provider / family | Models | Renderer classes |
| --- | --- | --- |
| [thinkingmachines](https://huggingface.co/thinkingmachines) | [Inkling](https://huggingface.co/thinkingmachines/Inkling), [Inkling-Small](https://huggingface.co/thinkingmachines/Inkling-Small) | `v0.Renderer` (TMLv0) |
| Qwen | Qwen3 (including vision), Qwen3.5, Qwen3.8 | `Qwen3`, `Qwen35`, `Qwen38` |
| DeepSeek | V3, R1, V3.1, V4, V4.1 Flash | `DeepSeekV3`, `DeepSeekR1`, `DeepSeekV31`, `DeepSeekV4`, `DeepSeekV41` |
| Kimi | K2, K2.5 | `KimiK2`, `KimiK25` |
| GLM | 5.2, 5.3, 5.3 Flash | `Glm52`, `Glm53`, `Glm53Flash` |
| GPT-OSS | GPT-OSS | `GptOss` |
| Nemotron | Nemotron 3 | `Nemotron3` |

## OpenAI message compatibility

The same conversation can use OpenAI-style dictionaries:

| Message | Native chat object | OpenAI-style dict |
| --- | --- | --- |
| User | `chat.Message(content=chat.Text("Hello, world!"), author=chat.Author(chat.AuthorKind.User))` | `{"role": "user", "content": "Hello, world!"}` |
| Assistant | `chat.Message(content=chat.Text("Hello!"), author=chat.Author(chat.AuthorKind.Model))` | `{"role": "assistant", "content": "Hello!"}` |

Convert the dictionaries before passing them to a renderer:

```py
from tml_renderers import chat, renderers

openai_dicts = [
    {"role": "user", "content": "Hello, world!"},
    {"role": "assistant", "content": "Hello!"},
]
openai_messages = chat.OpenAIMessage.from_oss_messages(openai_dicts)
round_tripped = chat.OpenAIMessage.to_oss_messages(openai_messages)

renderer = renderers.Qwen3()
training_examples = renderer.render_for_sft(openai_messages)
```

The renderer also accepts `openai_messages` for completion. The conversion
supports text, images, tool calls, and tool results; it handles message formats,
not the OpenAI network API.

## Rendering and parsing (TMLv0)

Rendering converts structured chat messages into the token and media-span
sequence consumed by the model. `v0` uses the same TMLv0 grammar for both
inference and training:

- `render_for_completion(...)` renders an inference prompt and returns a parser
  for turning generated model tokens back into messages.
- `render_for_sft(...)` renders supervised-training examples with aligned,
  per-token loss weights.

The parser reconstructs structured messages from generated token spans:

```py
from tml_renderers import chat, tokenizers, v0

tokenizer = tokenizers.O200kBaseChatTokenizer()
renderer = v0.Renderer(tokenizer)

messages = [
    chat.Message(
        content=chat.Text("Hello"),
        author=chat.Author(chat.AuthorKind.User),
    )
]

spans, parser = renderer.render_for_completion(messages)
parsed = parser.parse(spans)

training_examples = renderer.render_for_sft(messages)
```

The package includes native tokenizers for these renderers. It does not need
Hugging Face chat templates or remote code.

```py
tokenizer.encode_ordinary("hello")
tokenizer.decode([24912])
tokenizer.encode_special("message_user")   # -> 200000
tokenizer.decode_special(200000)            # -> "message_user"

qwen_tokenizer = tokenizers.Qwen3Tokenizer()
qwen_tokenizer.encode_special("im_start")   # -> 151644
```

Full method surface:

| method | maps | on invalid / partial UTF-8 |
| :---- | :---- | :---- |
| `encode_ordinary(text)` | text → ids | — (special-token markup is encoded as literal text) |
| `encode_special(name)` | name → id | raises if `name` isn't a special token |
| `decode(ids)` | ids → text | **lossy** — invalid/partial bytes become `U+FFFD`, never raises (tiktoken parity) |
| `decode_strict(ids)` | ids → text | **raises** `ValueError` on invalid/partial UTF-8 |
| `decode_bytes(ids)` | ids → bytes | raw bytes, no UTF-8 handling |
| `decode_special(id)` | id → name | `None` if `id` isn't a special token |

The tokenizer also provides `is_special_token(id)`, `special_tokens()`, and
`eos_token`, `bos_token`, and `all_special_tokens`. `decode` is **lossy**: a
truncated multibyte tail renders as `�` rather than raising. Use
`decode_strict` when invalid UTF-8 must raise an error.

`render_for_completion` accepts a `MessageList`, a `list[Message]`, or a
`list[OpenAIMessage]`. For TMLv0,
`render_for_completion_with_effort(messages, effort)` accepts an effort float in
`[0, 1]` and inserts a system message before the first non-system message;
input messages must not already contain `ThinkingEffort`. For SFT,
`render_for_sft` returns `TrainingExample`s with per-token loss weights.

### Multimodal inputs

`tml-renderers` includes multimodal codecs for the TMLv0 renderer:

- An `ImagePointer` is rendered as an `ImageAssetPointerTokenSpan` with the
  expected image-token count.
- An `AudioPointer` is rendered as a `DmelTokenSpan`.

Image and audio messages use the same completion and training APIs as text:

```py
from tml_renderers import chat, tokenizers, v0

renderer = v0.Renderer(tokenizers.O200kBaseChatTokenizer())

image_msg = chat.Message(
    content=chat.ImagePointer(
        location="image.png",
        format=chat.ImageFormat.Png,
        width=512,
        height=512,
    ),
    author=chat.Author(chat.AuthorKind.User),
)

audio_msg = chat.Message(
    content=chat.AudioPointer(
        location="clip.wav",
        format=chat.AudioFormat.Wav,
        num_frames=48_000,
        sample_rate=16_000,
    ),
    author=chat.Author(chat.AuthorKind.User),
)

spans, _ = renderer.render_for_completion([image_msg, audio_msg])
dmel = next(s.span for s in spans if type(s.span).__name__ == "DmelTokenSpan")
print(dmel.dmel.shape)  # [num_audio_tokens, num_dmel_bins], uint8
```

Media locations are represented as pointers. Audio is read from local files;
`tml-renderers` never fetches remote media.

## Token spans

Renderers work with token-span types you can construct directly:

```py
from tml_renderers.chat import (
    EncodedTextTokenSpan,
    ImageAssetPointerTokenSpan,
    ImageFormat,
    PaddingTokenSpan,
    TokenSpan,
)

text = EncodedTextTokenSpan([101, 102])
image = ImageAssetPointerTokenSpan(
    location="image.png", format=ImageFormat.Png, width=512, height=512, num_tokens=256
)
padding = PaddingTokenSpan(4)

# Wrap in the oneof only where a call boundary asks for a TokenSpan.
wrapped = TokenSpan(text)
```

`DmelTokenSpan` is the encoded-audio span produced by the renderer. Its
`dmel` field is a `uint8` tensor of shape
`[num_audio_tokens, num_dmel_bins]`.

## Using with Tinker

In Tinker Cookbook, use the recommended renderer for your model. Select
`tml_v0` for Inkling and Inkling-Small; supported third-party models use their
model-specific renderers. Cookbook converts rendered text, image, and audio
inputs for sampling and SFT.

The lower-level `tml_renderers.tinker` helpers are also available for direct
Tinker SDK integration. Provide local media files or base64 image data.

## Types

- **Core**: `Message`, `MessageList`, `Author`, `AuthorKind`,
  `MessageChannel`, `MessageMetadata`
- **Content**: `Text`, `Thinking`, `ThinkingEffort`, `ImagePointer`,
  `AudioPointer`, `InvokeTool`, `StructuredToolCall`, `ToolError`,
  `ToolDeclareJson`, `ModelEndSampling`
- **Token spans**: `TokenSpan`, `EncodedTextTokenSpan`,
  `ImageAssetPointerTokenSpan`, `DmelTokenSpan`, `PaddingTokenSpan`
- **Formats**: `ImageFormat`, `AudioFormat`
- **Tool args**: `ToolArg`, `ToolMetadata`. `ToolArg.value` is a JSON-encoded
  string, matching the OpenAI `function.arguments` convention.
- **Tool declarations**: `ToolSpecJson` (one declared tool with `name`,
  `description`, `type_name`, and JSON-schema `parameters`), grouped into
  `ToolDeclareJson`. The constructor accepts a dict for `parameters` and
  serializes it.

## Examples

- `examples/run_tml_v0.py` — low-level rendering and parsing of text and audio.
- `examples/cookbook_tinker_demo.py` — native and OpenAI-style messages,
  including tool calls and results. See the
  [public tool-call example](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/#tool-calls).

## License

Apache-2.0. The wheel includes `LICENSE`, `NOTICE`, and
`THIRD_PARTY_LICENSES.md`.

