Metadata-Version: 2.4
Name: tml-renderers
Version: 0.4.2
Classifier: Typing :: Typed
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: Apache Software License
License-File: LICENSE
License-File: NOTICE
License-File: THIRD_PARTY_LICENSES.md
Summary: Public TML chat types, tokenizers, and native renderers
Keywords: chat,message,tml
Author-email: Thinking Machines Lab <tinker@thinkingmachines.ai>
License-Expression: Apache-2.0
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://tinker-docs.thinkingmachines.ai/changelog/renderers/
Project-URL: Documentation, https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/
Project-URL: Homepage, https://thinkingmachines.ai/tinker

# tml-renderers

`tml-renderers` turns structured text, image, and audio chat into model-ready
tokens and media for training and inference. It supports Thinking Machines Lab
models, including Inkling, Inkling-Small, and Inkling 2 Small, through TMLv0, and the
[supported third-party models served on Tinker](https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/)
through model-specific renderers.

See the [changelog](https://tinker-docs.thinkingmachines.ai/changelog/renderers/)
for the changes in each release.

## Installation

Always use the latest release:

```sh
pip install --upgrade tml-renderers
```

`tml-renderers` is a compiled extension that requires PyTorch 2.11 or newer.

## Quickstart

Create a short conversation with native chat objects and render it for
inference or training:

```py
from tml_renderers import chat, renderers

user = chat.Message(
    content=chat.Text("Hello, world!"),
    author=chat.Author(chat.AuthorKind.User),
)
assistant = chat.Message(
    content=chat.Text("Hello!"),
    author=chat.Author(chat.AuthorKind.Model),
)
native_messages = chat.MessageList([user, assistant])

renderer = renderers.Qwen3()
spans, parser = renderer.render_for_completion([user])
training_examples = renderer.render_for_sft(native_messages)
```

To continue from the exact sampled token prefix, bridge the parsed semantic
history to the next user or tool messages instead of rerendering earlier turns:

```py
sampled = chat.TokenSpan(chat.EncodedTextTokenSpan(sampled_token_ids))
parsed = parser.parse_tokens(sampled_token_ids)
follow_up = chat.Message(
    content=chat.Text("Tell me more."),
    author=chat.Author(chat.AuthorKind.User),
)

spans, suffix, parser = renderer.bridge_to_next_turn(
    [*spans, sampled],
    [user, *parsed],
    [follow_up],
)
```

The first return value preserves every token in the supplied prefix and appends
`suffix`. The returned parser is configured for the combined semantic history.

## Supported models

| Provider / family | Models | Renderer classes |
| --- | --- | --- |
| [thinkingmachines](https://huggingface.co/thinkingmachines) | [Inkling](https://huggingface.co/thinkingmachines/Inkling), [Inkling-Small](https://huggingface.co/thinkingmachines/Inkling-Small) | `Inkling` (TMLv0) |
| [thinkingmachines](https://huggingface.co/thinkingmachines) | [Inkling 2 Small](https://huggingface.co/thinkingmachines/Inkling-2-Small) | `Inkling2` (TMLv0) |
| Qwen | Qwen3 (including vision), Qwen3.5, Qwen3.8 | `Qwen3`, `Qwen35`, `Qwen38` |
| DeepSeek | V3, V3.1, V4, V4.1 Flash | `DeepSeekV3`, `DeepSeekV31`, `DeepSeekV4`, `DeepSeekV41` |
| Kimi | K2, K2.5, K3.1-Pro | `KimiK2`, `KimiK25`, `KimiK31Pro` |
| GLM | 5.2, 5.3, 5.3 Flash | `Glm52`, `Glm53`, `Glm53Flash` |
| GPT-OSS | GPT-OSS | `GptOss` |
| Nemotron | Nemotron 3, Nemotron 3.5, Nemotron 3.5 Super VL | `Nemotron3`, `Nemotron3VL` |

`Glm52` clears reasoning from earlier turns. `Glm53` and `Glm53Flash` keep it. Pass
`clear_thinking=True` to clear it, as
[Z.ai recommends](https://huggingface.co/zai-org/GLM-5.3#note). Use the same setting for
rollout and SFT.

## OpenAI message compatibility

Convert OpenAI-style message dictionaries before passing them to a renderer:

```py
from tml_renderers import chat, renderers

openai_dicts = [
    {"role": "user", "content": "Hello, world!"},
    {"role": "assistant", "content": "Hello!"},
]
openai_messages = chat.OpenAIMessage.from_openai_chat_dicts(openai_dicts)
round_tripped = chat.OpenAIMessage.to_openai_chat_dicts(openai_messages)

renderer = renderers.Qwen3()
training_examples = renderer.render_for_sft(openai_messages)
```

The renderer also accepts `openai_messages` for completion. The conversion
supports text, reasoning, images, tool calls, and tool results; it handles message
formats, not the OpenAI network API. An assistant dict keeps its reasoning in a
top-level `reasoning_content`. As in an OpenAI request, tools are not a message:
pass the `tools` list as `chat.OpenAIMessage.from_openai_chat_dicts(openai_dicts,
tools=tools)`.

The `from_oss_*` and `to_oss_*` methods read and write tinker_cookbook message
dicts. They are deprecated.

## Rendering and parsing (TMLv0)

Rendering converts structured chat messages into the token and media-span
sequence consumed by the model. `renderers.Inkling` and `renderers.Inkling2` use
the same TMLv0 grammar for both inference and training:

- `render_for_completion(...)` renders an inference prompt and returns a parser
  for turning generated model tokens back into messages.
- `render_for_sft(...)` renders supervised-training examples with aligned,
  per-token loss weights.

The parser reconstructs structured messages from generated token spans:

```py
from tml_renderers import chat, renderers

renderer = renderers.Inkling()
tokenizer = renderer.tokenizer

messages = [
    chat.Message(
        content=chat.Text("Hello"),
        author=chat.Author(chat.AuthorKind.User),
    )
]

spans, parser = renderer.render_for_completion(messages)
parsed = parser.parse(spans)

training_examples = renderer.render_for_sft(messages)
```

For incremental sampling, open a parser stream and feed it only the newly sampled
token IDs. It returns the same messages as a batch parse of the same tokens:

```py
completed_messages = []
with parser.parse_stream() as stream:
    for token_ids in sampled_token_chunks:
        for update in stream.feed(token_ids):
            event = update.update
            if isinstance(event, chat.StreamingMessageHeader):
                print(f"[{event.author.kind}] ", end="")
            elif isinstance(event, chat.StreamingContent):
                print(event.content.text, end="", flush=True)
            elif isinstance(event, chat.Message):
                completed_messages.append(event)
    for update in stream.finish():
        if isinstance(update.update, chat.Message):
            completed_messages.append(update.update)
```

`finish()` validates the end of the stream and releases the parser. Leaving the
context unfinished discards partial state and releases the parser. Parse exceptions
expose `completed_messages`, which contains any complete prefix parsed before the
error.

The package includes native tokenizers for these renderers. It does not need
Hugging Face chat templates, remote code, or network access. `Inkling` uses
`tokenizers.O200kBaseChatTokenizer()` and `Inkling2` uses
`tokenizers.H200kBaseChatTokenizer()`. Both share the same special-token IDs.
`v0.Renderer(tokenizer)` accepts either tokenizer.

```py
from tml_renderers import tokenizers

tokenizer.encode_ordinary("hello")
tokenizer.decode([24912])
tokenizer.encode_special("message_user")   # -> 200000
tokenizer.decode_special(200000)            # -> "message_user"

qwen_tokenizer = tokenizers.Qwen3Tokenizer()
qwen_tokenizer.encode_special("im_start")   # -> 151644
```

Full method surface:

| method | maps | on invalid / partial UTF-8 |
| :---- | :---- | :---- |
| `encode_ordinary(text)` | text → ids | — (special-token markup is encoded as literal text) |
| `encode_special(name)` | name → id | raises if `name` isn't a special token |
| `decode(ids)` | ids → text | **lossy** — invalid/partial bytes become `U+FFFD`, never raises (tiktoken parity) |
| `decode_strict(ids)` | ids → text | **raises** `ValueError` on invalid/partial UTF-8 |
| `decode_bytes(ids)` | ids → bytes | raw bytes, no UTF-8 handling |
| `decode_special(id)` | id → name | `None` if `id` isn't a special token |

The tokenizer also provides `is_special_token(id)`, `special_tokens()`,
`eos_token`, `bos_token`, and `all_special_tokens`.

`render_for_completion` accepts a `MessageList`, a `list[Message]`, or a
`list[OpenAIMessage]`.

To render at one thinking effort, pass `effort` (a float in `[0, 1]`) to
`renderers.Inkling`, `renderers.Inkling2`, or `v0.Renderer`. The renderer then adds
the effort message to every completion and SFT render. Do not also put a
`ThinkingEffort` message in the input.

### Multimodal inputs

`tml-renderers` includes multimodal codecs for the TMLv0 renderer:

- An `ImagePointer` is rendered as an `ImageAssetPointerTokenSpan` with the
  expected image-token count.
- An `AudioPointer` is rendered as a `DmelTokenSpan`.

Image and audio messages use the same completion and training APIs as text:

```py
from tml_renderers import chat, renderers

renderer = renderers.Inkling()

image_msg = chat.Message(
    content=chat.ImagePointer(
        location="image.png",
        format=chat.ImageFormat.Png,
        width=512,
        height=512,
    ),
    author=chat.Author(chat.AuthorKind.User),
)

audio_msg = chat.Message(
    content=chat.AudioPointer(
        location="clip.wav",
        format=chat.AudioFormat.Wav,
        num_frames=48_000,
        sample_rate=16_000,
    ),
    author=chat.Author(chat.AuthorKind.User),
)

spans, _ = renderer.render_for_completion([image_msg, audio_msg])
dmel = next(s.span for s in spans if type(s.span).__name__ == "DmelTokenSpan")
print(dmel.dmel.shape)  # [num_audio_tokens, num_dmel_bins], uint8
```

Media locations are represented as pointers. Audio is read from local files;
`tml-renderers` never fetches remote media.

## Token spans

Renderers work with token-span types you can construct directly:

```py
from tml_renderers.chat import (
    EncodedTextTokenSpan,
    ImageAssetPointerTokenSpan,
    ImageFormat,
    PaddingTokenSpan,
    TokenSpan,
)

text = EncodedTextTokenSpan([101, 102])
image = ImageAssetPointerTokenSpan(
    location="image.png", format=ImageFormat.Png, width=512, height=512, num_tokens=256
)
padding = PaddingTokenSpan(4)

# Wrap in the oneof only where a call boundary asks for a TokenSpan.
wrapped = TokenSpan(text)
```

`DmelTokenSpan` is the encoded-audio span produced by the renderer. Its
`dmel` field is a `uint8` tensor of shape
`[num_audio_tokens, num_dmel_bins]`.

`chat.DmelTokenSpan(tensor, target_audio_token=...)` copies a 2-D CPU `torch.uint8`
tensor. The `.dmel` getter returns a tensor that shares the span's storage, so writes
to it change the span.

## Using with Tinker

In Tinker Cookbook, use the recommended renderer for your model. Cookbook
converts rendered text, image, and audio inputs for sampling and SFT.

The lower-level `tml_renderers.tinker` helpers are also available for direct
Tinker SDK integration. Provide local media files or base64 image data.

## Types

- **Core**: `Message`, `MessageList`, `Author`, `AuthorKind`,
  `MessageChannel`, `MessageMetadata`
- **Content**: `Text`, `Thinking`, `ThinkingEffort`, `ImagePointer`,
  `AudioPointer`, `InvokeTool`, `StructuredToolCall`, `ToolError`,
  `ToolDeclareJson`, `ModelEndSampling`
- **Token spans**: `TokenSpan`, `EncodedTextTokenSpan`,
  `ImageAssetPointerTokenSpan`, `DmelTokenSpan`, `PaddingTokenSpan`
- **Formats**: `ImageFormat`, `AudioFormat`
- **Tool args**: `ToolArg`, `ToolMetadata`. `ToolArg.value` is a JSON-encoded
  string, matching the OpenAI `function.arguments` convention.
- **Tool declarations**: `ToolSpecJson` (one declared tool with `name`,
  `description`, `type_name`, and JSON-schema `parameters`), grouped into
  `ToolDeclareJson`. The constructor accepts a dict for `parameters` and
  serializes it.

## Examples

- `examples/run_tml_v0.py` — low-level rendering and parsing of text and audio.
- `examples/cookbook_tinker_demo.py` — native and OpenAI-style messages,
  including tool calls and results. See the
  [public tool-call example](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/#tool-calls).

## License

Apache-2.0. The wheel includes `LICENSE`, `NOTICE`, and
`THIRD_PARTY_LICENSES.md`.

