Metadata-Version: 2.4
Name: dspy-toon
Version: 0.4.1
Summary: DSPy adapter using TOON (Token-Oriented Object Notation) v4.1 format for compact structured LLM inputs and outputs
Project-URL: Homepage, https://github.com/Archelunch/dspy-toon
Project-URL: Repository, https://github.com/Archelunch/dspy-toon
Project-URL: Documentation, https://github.com/Archelunch/dspy-toon#readme
Project-URL: Bug Tracker, https://github.com/Archelunch/dspy-toon/issues
Author-email: Michael Pavlukhin <michael@pavlukhinlab.com>
License-Expression: MIT
License-File: LICENSE
License-File: NOTICE
Keywords: adapter,ai,dspy,llm,machine-learning,nlp,prompt-engineering,serialization,serialization-format,structured-output,token-efficient,token-reduction,toon,toon-format
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.10
Requires-Dist: dspy<4,>=3.3.1
Requires-Dist: pydantic>=2.0.0
Provides-Extra: all
Requires-Dist: datasets>=4.4.1; extra == 'all'
Requires-Dist: httpx>=0.28.1; extra == 'all'
Requires-Dist: jsonschema>=4.0.0; extra == 'all'
Requires-Dist: mypy>=1.8.0; extra == 'all'
Requires-Dist: pandas>=2.0.0; extra == 'all'
Requires-Dist: pyarrow>=21.0.0; extra == 'all'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'all'
Requires-Dist: pytest-cov>=4.1.0; extra == 'all'
Requires-Dist: pytest>=8.0.0; extra == 'all'
Requires-Dist: ruff>=0.8.0; extra == 'all'
Requires-Dist: tabulate>=0.9.0; extra == 'all'
Requires-Dist: tiktoken>=0.4.0; extra == 'all'
Provides-Extra: benchmark
Requires-Dist: datasets>=4.4.1; extra == 'benchmark'
Requires-Dist: httpx>=0.28.1; extra == 'benchmark'
Requires-Dist: jsonschema>=4.0.0; extra == 'benchmark'
Requires-Dist: pandas>=2.0.0; extra == 'benchmark'
Requires-Dist: pyarrow>=21.0.0; extra == 'benchmark'
Requires-Dist: tabulate>=0.9.0; extra == 'benchmark'
Requires-Dist: tiktoken>=0.4.0; extra == 'benchmark'
Provides-Extra: dev
Requires-Dist: mypy>=1.8.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.8.0; extra == 'dev'
Description-Content-Type: text/markdown

# DSPy-TOON

[![Tests](https://github.com/Archelunch/dspy-toon/actions/workflows/test.yml/badge.svg)](https://github.com/Archelunch/dspy-toon/actions)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Buy Me a Coffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-orange?logo=buy-me-a-coffee)](https://buymeacoffee.com/mike_pavlukhin)

DSPy-TOON is a [DSPy](https://dspy.ai/) adapter for structured LLM inputs and outputs in [TOON](https://github.com/toon-format/spec). Define your fields with DSPy signatures and Pydantic models, configure `ToonAdapter`, and use your DSPy predictors as usual. The adapter handles prompt formatting, serialization and output validation.

The package also exposes its Python encoder and decoder for applications that need TOON without running a predictor. It maintains its own codec and targets TOON 4.1.

TOON puts repeated field names in a shared header. This can reduce token use for lists of records, uniform nested objects and dictionaries of records. Savings depend on the data and the complete prompt: compact JSON can be smaller for irregular objects, and generating a compact format does not guarantee correct output.

## Contents

- [Installation](#installation)
- [Quick start](#quick-start)
- [How the adapter works](#how-the-adapter-works)
- [Typed outputs](#typed-outputs)
- [TOON format](#toon-format)
- [Encoder and decoder API](#encoder-and-decoder-api)
- [Async calls](#async-calls)
- [Streaming](#streaming)
- [Benchmarks](#benchmarks)
- [Upgrading to 0.4.0](#upgrading-to-040)
- [Development and contributions](#development-and-contributions)

## Installation

Requirements: Python 3.10 through 3.14, DSPy `>=3.3.1,<4`, and Pydantic 2. Local validation for this release used Python 3.12 and DSPy 3.3.1.

```bash
pip install dspy-toon
```

To install from source:

```bash
pip install "git+https://github.com/Archelunch/dspy-toon.git"
```

DSPy and Pydantic are the direct runtime dependencies. Pip also installs their dependencies. The wheel contains `dspy_toon`, type information and license notices; experiments, datasets, tests and blog assets stay out of the package. Benchmark dependencies are optional and documented under [development](#development-and-contributions).

## Quick start

Set the credentials for your model provider. This example uses `OPENAI_API_KEY`:

```python
import dspy
from pydantic import BaseModel, Field
from dspy_toon import ToonAdapter


class UserInfo(BaseModel):
    name: str = Field(description="Full name")
    age: int = Field(ge=0, description="Age in years")
    occupation: str = Field(description="Job title")


class ExtractUser(dspy.Signature):
    """Extract user information from text."""

    text: str = dspy.InputField()
    user: UserInfo = dspy.OutputField()


lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm, adapter=ToonAdapter())

extractor = dspy.Predict(ExtractUser)
result = extractor(text="Alice Johnson is a 35-year-old software engineer.")

print(result.user.name)
print(result.user.model_dump())
```

`result.user` is a validated `UserInfo` instance. Field descriptions guide the model; constraints such as `ge=0` validate its answer.

To select the adapter for a block of calls, use DSPy's context manager:

```python
with dspy.context(adapter=ToonAdapter()):
    result = extractor(text="Bob is 28 and works as a designer.")
```

Your model configuration stays in DSPy. The adapter does not configure provider credentials or run a model server.

## How the adapter works

The prompt includes enum choices, numeric bounds, length constraints and nullable alternatives, including fields inside lists and dictionaries. Syntax guidance follows the input and output types, and the final reminder names the expected output fields. For arrays, the prompt asks the model to quote each string item or cell so embedded commas cannot split values.

For each prediction, `ToonAdapter`:

1. Describes the signature's input and output fields, including types and nested field descriptions.
2. Adds TOON syntax instructions and output shape examples generated by the encoder.
3. Serializes the input fields and demonstration outputs as complete TOON objects.
4. Parses the model's response and validates each output against its signature annotation with Pydantic.

Both input and output formatting change when you switch adapters. To serialize only a data payload for an otherwise unchanged prompt, call `encode()` directly.

The adapter inherits DSPy's synchronous and asynchronous call paths and demo handling. It supports `dspy.History`; legacy lists of user/assistant pairs are also accepted. Review serialized demonstrations and saved optimizer artifacts when changing adapter or format versions. The upgrade does not establish optimizer-specific quality improvements.

### Output validation and errors

Responses must contain the signature's output fields. Unexpected fields and missing required fields raise `AdapterParseError`. Missing defaulted or nullable output fields follow DSPy 3.3.1's output-default handling.

The parser accepts a complete TOON document, a complete JSON object as a fallback, or a document enclosed in a code fence. It does not recover arbitrary fragments from surrounding prose. JSON fallback still goes through the same field validation.

```python
from dspy.utils.exceptions import AdapterParseError

try:
    result = extractor(text="Alice is 35 and works as an engineer.")
except AdapterParseError as error:
    print(error)
    # Decide whether to retry, ask for clarification, or reject the result.
```

Validation follows Pydantic's normal coercion rules unless your annotations request strict types. It checks the declared types and constraints, not whether the extracted facts match the source. Malformed TOON, wrong array lengths and duplicate keys fail during decoding.

To inspect the actual prompt without making a model call:

```python
adapter = ToonAdapter()
messages = adapter.format(
    signature=ExtractUser,
    demos=[],
    inputs={"text": "Alice is 35 and works as an engineer."},
)
for message in messages:
    print(message["role"], message["content"], sep="\n")
```

This is useful when comparing adapter overhead or checking how descriptions and nested inputs reach the model.

## Typed outputs

The examples below use the model configuration from the quick start.

### Lists of records

Uniform records share a tabular header in TOON. Define the output as a list of Pydantic models and iterate over the returned models normally:

```python
class ExtractPeople(dspy.Signature):
    """Extract every person mentioned in the text."""

    text: str = dspy.InputField()
    people: list[UserInfo] = dspy.OutputField()


extract_people = dspy.Predict(ExtractPeople)
result = extract_people(
    text="Alice is a 35-year-old engineer. Bob is a 28-year-old designer."
)
for person in result.people:
    print(person.name, person.age, person.occupation)
```

The same model types can be used in input fields. The encoder handles Pydantic models inside lists and dictionaries recursively.

### Nested models and nullable fields

```python
from typing import Literal


class Address(BaseModel):
    street: str
    city: str
    country: Literal["US", "UK", "DE"]


class UserProfile(BaseModel):
    name: str = Field(description="Full name")
    email: str = Field(description="Email address")
    address: Address | None = Field(
        description="Home address, or null if the text does not provide it"
    )


class ExtractProfile(dspy.Signature):
    """Extract a profile. Use null for an unknown address."""

    text: str = dspy.InputField()
    profile: UserProfile = dspy.OutputField()


extract_profile = dspy.Predict(ExtractProfile)
result = extract_profile(text="Contact John at john@example.com.")
print(result.profile)
```

`Address | None` permits an explicit `null`. Within a Pydantic model, a nullable field without a default is still required; add `= None` when omission should also be accepted.

### Classification with constraints

```python
class SentimentResult(BaseModel):
    sentiment: Literal["positive", "negative", "neutral"]
    confidence: float = Field(ge=0, le=1, description="Confidence from 0 to 1")
    key_phrases: list[str] = Field(description="Phrases supporting the label")


class AnalyzeSentiment(dspy.Signature):
    """Classify the sentiment of the text."""

    text: str = dspy.InputField()
    result: SentimentResult = dspy.OutputField()


analyzer = dspy.Predict(AnalyzeSentiment)
result = analyzer(text="I love this product. It works exactly as promised.")
print(result.result.sentiment, result.result.confidence)
```

The adapter renders union and `Literal` annotations in its type guidance and validates them on return. A valid confidence value is not a calibrated probability merely because it lies between 0 and 1.

## TOON format

These are serialization examples. The full adapter prompt also includes task instructions, field descriptions and format guidance.

### Flat records

JSON repeats the keys:

```json
{"people":[{"name":"Alice","age":35},{"name":"Bob","age":28}]}
```

TOON lists them once:

```text
people[2]{name,age}:
  Alice,35
  Bob,28
```

### Nested field groups

Uniform nested objects can share a header as well:

```text
people[2]{name,address{city,country}}:
  Alice,Paris,FR
  Bob,Berlin,DE
```

Each row reconstructs an object with `name` and a nested `address` object.

### Keyed tables

Dictionaries of uniform records retain their keys in keyed rows:

```text
people[2:]{age,city}:
  alice: 35,Paris
  bob: 28,Berlin
```

This decodes to `{"people": {"alice": {"age": 35, "city": "Paris"}, "bob": {"age": 28, "city": "Berlin"}}}`.

### Mixed structures and empty values

Arrays that do not fit a shared table use list items. With the default indentation, rows under a field on a list-item line are four spaces deeper than the hyphen:

```text
items[2]:
  - users[2]{id,name}:
      1,Alice
      2,Bob
    status: active
  - users[1]{id,name}:
      3,Carol
    status: pending
```

Empty arrays use `items: []`. Nullable values use `address: null`. Strings that resemble numbers, booleans or null are quoted to preserve their type. The encoder also quotes delimiter-sensitive strings and escapes control characters.

TOON 4.1 is the working draft reviewed for this release. See the pinned specification and compatibility notes in [UPGRADE.md](https://github.com/Archelunch/dspy-toon/blob/main/UPGRADE.md).

## Encoder and decoder API

```python
from dspy_toon import ToonDecodeError, decode, encode

records = {"people": [{"name": "Alice", "age": 35}, {"name": "Bob", "age": 28}]}
text = encode(records)
assert decode(text) == records
```

| Export | Purpose |
|---|---|
| `ToonAdapter()` | DSPy prompt formatting and validated output parsing |
| `encode(value, options=None)` | Convert a Python value to TOON text |
| `decode(text, options=None)` | Decode TOON into Python dictionaries, lists and scalar values |
| `ToonDecodeError` | `ValueError` subclass for malformed TOON |
| `enable_toon_streaming()` | Install the optional scalar streaming integration |
| `is_streaming_enabled()` | Report whether that integration is installed |

### Codec options

Options are passed as a dictionary:

```python
text = encode(records, {"indentSize": 4, "delimiter": "|"})
restored = decode(text, {"indentSize": 4, "strict": True})
assert restored == records
```

| Option | Applies to | Default | Behavior |
|---|---|---|---|
| `indentSize` | Encode and decode | `2` | Positive integer specifying the indentation step |
| `indent` | Encode and decode | `2` | Compatibility alias; `indentSize` takes precedence |
| `delimiter` | Encode | `","` | Comma, tab (`"\t"`) or pipe (`"|"`); the decoder reads it from the header |
| `strict` | Decode | `True` | Validate structure, counts, widths and duplicate keys |

The adapter uses the codec defaults. These options configure standalone `encode()` and `decode()` calls; `ToonAdapter` does not expose a codec-options argument.

Keep strict decoding enabled for model output. Non-strict decoding tolerates count and width mismatches, keeps the last duplicate key, and relaxes indentation. It can omit missing cells or ignore extra cells, so it is unsuitable as a general repair step when data integrity matters. The obsolete `lengthMarker` encoding option raises an error.

### Python value conversion

Pydantic models use `model_dump()` recursively. Dictionaries become objects, tuples and sets become arrays, and dates use ISO 8601 strings. Dictionary keys become strings. `decode()` returns plain Python values; the adapter reconstructs typed outputs separately.

Integers retain Python precision. Decimal inputs use float approximation, and non-finite float values encode as `null`. Unsupported Python objects also encode as `null`, so normalize custom values before encoding them. The codec is a data serializer, not a lossless serializer for arbitrary Python objects.

```python
try:
    decode("people[2]{name,age}:\n  Alice,35")
except ToonDecodeError as error:
    print(error)  # The document declares two rows but contains only one.
```

## Async calls

Use `acall()` for an asynchronous predictor call:

```python
import asyncio


async def main():
    result = await extractor.acall(
        text="Alice Johnson is a 35-year-old software engineer."
    )
    print(result.user)


asyncio.run(main())
```

Inside an existing event loop, call `await extractor.acall(...)` directly. DSPy's `asyncify()` remains available for synchronous programs:

```python
async_extractor = dspy.asyncify(extractor)
# Inside an async function:
# result = await async_extractor(text="Alice is a 35-year-old engineer.")
```

## Streaming

Enable the integration before creating stream listeners. Choose a scalar output field, such as a text answer, for incremental display:

```python
import asyncio
import dspy
from dspy_toon import ToonAdapter, enable_toon_streaming, is_streaming_enabled


enable_toon_streaming()
assert is_streaming_enabled()

dspy.configure(
    lm=dspy.LM("openai/gpt-4o-mini", cache=False),
    adapter=ToonAdapter(),
)
predict = dspy.Predict("question -> answer")
stream_predict = dspy.streamify(
    predict,
    stream_listeners=[
        dspy.streaming.StreamListener(signature_field_name="answer")
    ],
)


async def stream_response():
    async for chunk in stream_predict(question="Explain how a hash table works."):
        if isinstance(chunk, dspy.streaming.StreamResponse):
            print(chunk.chunk, end="", flush=True)
        elif isinstance(chunk, dspy.Prediction):
            print("\nFinal answer:", chunk.answer)


asyncio.run(stream_response())
```

For synchronous iteration, create the stream with `async_streaming=False`:

```python
stream_predict = dspy.streamify(
    predict,
    stream_listeners=[
        dspy.streaming.StreamListener(signature_field_name="answer")
    ],
    async_streaming=False,
)
for chunk in stream_predict(question="What is a hash collision?"):
    if isinstance(chunk, dspy.streaming.StreamResponse):
        print(chunk.chunk, end="", flush=True)
    elif isinstance(chunk, dspy.Prediction):
        print("\nFinal answer:", chunk.answer)
```

Incremental chunks contain raw TOON text, including string quotes and escape sequences. Use the final `Prediction` for decoded, validated values. Arrays, tables and keyed objects arrive in that final prediction rather than as incrementally decoded records.

`enable_toon_streaming()` is idempotent and patches DSPy's internal `StreamListener` methods for the process. Calls using other adapters delegate to the original methods. This integration was checked with DSPy 3.3.1; recheck it when upgrading DSPy because the listener internals are not a stable registration API.

See DSPy's [streaming documentation](https://dspy.ai/tutorials/streaming/) for listener and program configuration.

## Benchmarks

Measure the complete request and response for your workload. Encoder-only token counts omit format instructions, schemas and demonstrations. Model runs also need to account for reasoning tokens, parse failures and answer quality.

Our Qwen3.8-27B-FP8 experiments compare adapter behavior and separate input serialization from output generation:

| Experiment | Scope | Report |
|---|---|---|
| Initial adapter comparison | TOON, JSON and chat adapters with reasoning on and off | [Findings](https://github.com/Archelunch/dspy-toon/blob/main/benchmark_results/qwen_20260905/FINDINGS.md) |
| Expanded comparison | 4,376 scored calls across 544 fresh cases, including input/output format ablations | [Expanded report](https://github.com/Archelunch/dspy-toon/blob/main/benchmark_results/qwen_blog_expanded/BLOG_DATA.md) |
| Input-format study | Upstream reading tasks and synthetic tool-result workloads with controlled outputs | [Interpretation](https://github.com/Archelunch/dspy-toon/blob/main/benchmark_results/qwen_input_study/INTERPRETATION.md) |

The input study found lower total token use for uniform inventory, nested customer records and keyed service maps when the model returned JSON. Irregular logs and nested order arrays used more tokens. Parsing policy changed some apparent accuracy differences. These are measurements from one model deployment, not a general accuracy or savings guarantee.

Reports include task definitions, sample sizes, uncertainty and failure accounting. Raw model payloads remain in a local archive pending publication review; see [data availability](https://github.com/Archelunch/dspy-toon/blob/main/benchmark_results/DATA_AVAILABILITY.md). The [blog draft](https://github.com/Archelunch/dspy-toon/blob/main/blog/toon-vs-json-in-dspy.md) discusses the experiments and charts in more detail.

## Upgrading to 0.4.0

- Upgrade readers before writers if you exchange TOON with another system. Older codecs may not understand nested field groups and keyed tables.
- Regenerate stored prompts and demos that depend on the earlier syntax. Empty arrays now encode as `[]`; the decoder still accepts legacy `[0]:` forms.
- Re-encode affected old list-item tables with the previous reader and the new writer. Strict decoding rejects the old codec's under-indented rows.
- Review hash-leading strings in stored v3 documents. Full lines starting with `#` are comments in v4; the new encoder quotes those strings.
- Handle `AdapterParseError` where callers previously relied on permissive parsing. Outputs now pass through complete-document parsing and Pydantic validation.
- Install the benchmark extra explicitly if your scripts use dataset tooling.

See [UPGRADE.md](https://github.com/Archelunch/dspy-toon/blob/main/UPGRADE.md) for the pinned specification, migration details and verification results, and the [changelog](https://github.com/Archelunch/dspy-toon/blob/main/CHANGELOG.md) for the release history.

## Development and contributions

```bash
git clone https://github.com/Archelunch/dspy-toon.git
cd dspy-toon
pip install -e ".[dev]"

pytest tests/ -v
mypy src/
ruff check src/ tests/
ruff format src/ tests/
```

For coverage, run `pytest tests/ --cov=dspy_toon --cov-report=term`.

To run experiments from a checkout, add their optional dependencies:

```bash
pip install -e ".[benchmark]"
python -m benchmarks.adapter_comparison --model gemini/gemini-2.5-flash-lite
```

The benchmark extra installs dependencies, not benchmark modules into site-packages. Scripts, datasets and reports belong to the repository. Individual experiments may need additional setup; consult their protocols before running them.

The library has four modules: `adapter.py` for DSPy integration, `toon.py` for serialization, `streaming.py` for the optional listener integration, and `__init__.py` for public exports. The [examples directory](https://github.com/Archelunch/dspy-toon/tree/main/examples) contains extraction, tabular-data and nested-model examples.

See [CONTRIBUTING.md](https://github.com/Archelunch/dspy-toon/blob/main/CONTRIBUTING.md) for contribution guidelines. Include a reproducible input and the expected output when reporting a codec or adapter issue.

## License

MIT. See [LICENSE](https://github.com/Archelunch/dspy-toon/blob/main/LICENSE) and [NOTICE](https://github.com/Archelunch/dspy-toon/blob/main/NOTICE) for copyright and attribution notices.
