Metadata-Version: 2.4
Name: polyrt
Version: 0.1.0
Summary: Unified Python interface for LLMs across local and cloud backends.
Project-URL: Homepage, https://github.com/ramankrishna/polyRT
Project-URL: Documentation, https://github.com/ramankrishna/polyRT#readme
Project-URL: Issues, https://github.com/ramankrishna/polyRT/issues
Author: Rama Krishna Bachu
License: Apache-2.0
License-File: LICENSE
Keywords: anthropic,inference,llm,mlx,openai,structured-outputs
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: pydantic>=2.6
Requires-Dist: typing-extensions>=4.10
Provides-Extra: all
Requires-Dist: anthropic>=0.40; extra == 'all'
Requires-Dist: mlx-lm>=0.20; extra == 'all'
Requires-Dist: mlx>=0.20; extra == 'all'
Requires-Dist: openai>=1.50; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: jsonschema>=4.21; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: twine>=5.1; extra == 'dev'
Provides-Extra: mlx
Requires-Dist: mlx-lm>=0.20; extra == 'mlx'
Requires-Dist: mlx>=0.20; extra == 'mlx'
Provides-Extra: openai
Requires-Dist: openai>=1.50; extra == 'openai'
Description-Content-Type: text/markdown

# polyrt

Unified Python interface for LLMs across local and cloud backends.

polyrt is a typed adapter library that exposes a single small surface — `generate`,
`stream`, and their async variants — backed by Anthropic's Claude, OpenAI's GPT
models, and local Apple-Silicon models via MLX. It does not route, it does not
orchestrate, it does not run a server: it gives you one consistent function call
shape and one consistent return type, regardless of which backend you point it at.

[![PyPI](https://img.shields.io/pypi/v/polyrt.svg)](https://pypi.org/project/polyrt/)
[![Python](https://img.shields.io/pypi/pyversions/polyrt.svg)](https://pypi.org/project/polyrt/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)

## Install

| Backend | Install |
| --- | --- |
| Claude (Anthropic) | `pip install polyrt[anthropic]` |
| OpenAI | `pip install polyrt[openai]` |
| MLX (Apple Silicon) | `pip install polyrt[mlx]` |
| All of the above | `pip install polyrt[all]` |

Python 3.11+ required.

## Quickstart

### Sync

```python
"""Synchronous one-shot generation."""

import polyrt

result = polyrt.generate(
    "claude",
    model="claude-haiku-4-5",
    messages=[{"role": "user", "content": "Say hello in one word."}],
    max_tokens=20,
)
print(result.text)
print(f"Tokens: {result.usage.input_tokens} in, {result.usage.output_tokens} out")
```

### Async streaming

```python
"""Async streaming generation."""

import asyncio

import polyrt


async def main() -> None:
    async for event in polyrt.astream(
        "openai",
        model="gpt-4.1-mini",
        messages=[{"role": "user", "content": "Count to five."}],
        max_tokens=40,
    ):
        if event.type == "token":
            print(event.delta, end="", flush=True)
        elif event.type == "done" and event.usage is not None:
            print(f"\n\n[{event.usage.total_tokens} tokens]")


asyncio.run(main())
```

### Structured output

```python
"""Schema-constrained structured output."""

from pydantic import BaseModel

import polyrt


class City(BaseModel):
    name: str
    country: str
    population: int


result = polyrt.generate(
    "claude",
    model="claude-haiku-4-5",
    messages=[{"role": "user", "content": "Pick a notable city and return its data."}],
    schema=City,
)
assert isinstance(result.parsed, City)
print(f"{result.parsed.name}, {result.parsed.country} — pop {result.parsed.population:,}")
```

## Backends

| Backend | Capabilities | Extra |
| --- | --- | --- |
| `claude` | schema, tools, vision, streaming | `polyrt[anthropic]` |
| `openai` | schema, tools, vision, streaming | `polyrt[openai]` |
| `mlx` | streaming (schema/tools/vision come in v0.2) | `polyrt[mlx]` |

## Schema enforcement

When you pass `schema=SomeModel`, polyrt asks the backend to constrain its output
to that Pydantic model and parses the result for you. The strategy differs by
backend:

* **Claude** — passes the schema as a single tool with `tool_choice` forced to
  that tool, then parses the tool's `input` payload.
* **OpenAI** — uses Structured Outputs (`response_format` with
  `type=json_schema`, `strict=true`) so the model is constrained at decode time.
* **MLX** — not yet supported in v0.1; raises `NotSupportedError`. v0.2 will add
  support via `outlines` or `lm-format-enforcer`.

`Response.parsed` is your validated Pydantic instance when it works, `None` otherwise.

## What polyrt is not

* Not a router that picks a backend for you
* Not an agent framework
* Not a model server or proxy
* Not a fine-tuning toolkit
* Not a prompt template engine

It is a typed adapter. If you want one of the things above, this is the layer
you would build it on top of.

## Adding a backend

1. Subclass `Backend` (or duck-type the protocol) and implement the seven methods.
2. Call `polyrt.registry.register("<name>", YourFactory)` at module import time.
3. Either ship inside `polyrt/backends/<name>.py`, or expose your factory as a
   `polyrt.backends` entry point so users get it via `pip install`.

See [docs/backends.md](docs/backends.md) for the full contract.

## License

Apache 2.0. See [LICENSE](LICENSE).

## Acknowledgments

* [`anthropic-sdk-python`](https://github.com/anthropics/anthropic-sdk-python)
* [`openai-python`](https://github.com/openai/openai-python)
* [`mlx-lm`](https://github.com/ml-explore/mlx-lm)
* [`litellm`](https://github.com/BerriAI/litellm) — prior art for the unified-interface idea.
