Metadata-Version: 2.4
Name: genai-pyo3
Version: 0.7.0.4
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: License :: OSI Approved :: MIT License
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: typing-extensions>=4.6.0
License-File: LICENSE-APACHE
License-File: LICENSE-MIT
Summary: Python bindings for rust-genai
License: MIT OR Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/ropoctl/genai-pyo3/issues
Project-URL: Repository, https://github.com/ropoctl/genai-pyo3

# genai-pyo3

Typed Python bindings for the Rust [`genai`](https://github.com/jeremychone/rust-genai) crate, built with `pyo3` and `maturin`.

This repo uses the upstream GitHub repository for `genai` directly:

```toml
genai = { git = "https://github.com/jeremychone/rust-genai" }
```

## What It Exposes

**Routing** — `AdapterKind`, `ModelIden`, `Endpoint`, `AuthData`, `ServiceTarget`

**Client** — `Client`, `ClientBuilder`

**Requests** — `ChatRequest`, `ChatMessage`, `ChatOptions`, `Tool`, `ToolCall`, `JsonSpec`, `Binary`

**Responses** — `ChatResponse`, `StreamEnd`, `ChatStreamEvent`, `Usage` (with prompt/completion/cache-creation detail)

**Embeddings** — `EmbedOptions`, `EmbedResponse`, `Embedding`

**Errors** — `GenaiError` and its subclasses (see below)

Entry points:

| | |
| --- | --- |
| `await client.achat(model, request, options=None)` | a full response |
| `await client.astream_chat(model, request, options=None)` | an async iterator of events |
| `await client.achat_via_stream(model, request, options=None)` | stream under the hood, one response out |
| `client.chat(model, request, options=None)` | blocking |
| `await client.aembed(model, text, options=None)` | one embedding |
| `await client.aembed_batch(model, texts, options=None)` | many |
| `client.resolve_service_target(model)` | where a call would go, without making it |
| `await client.aall_model_names(adapter_kind)` | what a provider is serving |

`model` is `str | ModelIden | ServiceTarget` everywhere.

## Install

Editable install with `uv`:

```bash
uv pip install -e .
```

This builds the `pyo3` extension and makes `genai_pyo3` importable from the active environment.

## Quick Start

```python
import asyncio
from genai_pyo3 import AuthData, ChatMessage, ChatRequest, Client


async def main() -> None:
    client = (
        Client.builder()
        .adapter_kind("openai")
        .provider("openai", auth=AuthData.from_env("OPENAI_API_KEY"))
        .build()
    )
    request = ChatRequest(messages=[ChatMessage("user", "Say hello in one short sentence")])

    response = await client.achat("gpt-4o-mini", request)
    print(response.text)


asyncio.run(main())
```

## Named endpoints

A `ServiceTarget` states where to go, how to authenticate, and which model to
ask for. A registry of them is just a dict, and one client serves all of it:

```python
from genai_pyo3 import AuthData, Client, ModelIden, ServiceTarget

client = Client()
registry = {
    "flash": ServiceTarget(
        "https://openrouter.ai/api/v1/",
        AuthData.from_env("OPENROUTER_API_KEY"),
        ModelIden("openai", "deepseek-ai/DeepSeek-V4-Flash-0731"),
    ),
    "local": ServiceTarget(
        "http://localhost:8000/v1/",
        AuthData.none(),
        ModelIden("openai", "Qwen/Qwen3.8-27B"),
    ),
}

response = await client.achat(registry["flash"], request)
```

`AuthData` never reveals a key to Python and redacts it in `repr()`, so
logging a `ServiceTarget` does not log a credential.

## Errors

```text
GenaiError(RuntimeError)
├── ConfigError        malformed request/options, adapter mismatch
├── AuthError          missing or unresolvable credentials
├── ResolverError      a resolver (including a Python callback) failed
├── ApiError           the provider answered with a failure status
│   ├── BadRequestError    4xx other than 429
│   ├── RateLimitError     429
│   └── ServerError        5xx
├── TransportError     connect/timeout/socket failures, dropped streams
├── StreamError        malformed or error events mid-stream
├── ResponseError      a well-formed reply that could not be interpreted
└── UnsupportedError   feature unavailable on this adapter/model
```

Every instance carries:

| | |
| --- | --- |
| `kind` | the precise failure in rust-genai's own vocabulary — the error variant in snake_case (`"http_error"`, `"no_auth_data"`), or the provider's own error type for a mid-stream failure (`"overloaded_error"`) |
| `status` | HTTP status, when the provider answered with one |
| `status_code` | compatibility alias for `status` |
| `retryable` | conservative hint; the retry and backoff policy is yours |
| `body` | response body, when there was one |
| `headers` | response headers, when the failure carried an HTTP response |
| `retry_after` | seconds, parsed from `headers` when present |
| `model_iden` | the model the failure is about, when the error names one |

The exception class is the actionable category; `kind` is the exact cause.
Branch on the class, log the `kind`.

```python
try:
    response = await client.achat(target, request)
except RateLimitError as err:
    await asyncio.sleep(err.retry_after or 5.0)
```

## Truncation

`stop_reason` is normalised across providers — `"completed"`,
`"max_tokens"`, `"tool_call"`, `"content_filter"`, `"stop_sequence"`,
`"other"` — with the provider's own wording kept in `stop_reason_raw`.
Without it, a truncated answer is indistinguishable from a short one.

## Images and PDFs

```python
from genai_pyo3 import Binary, ChatMessage

ChatMessage("user", content_parts=["what is in this?", Binary.from_path("chart.png")])
```

## Transport

```python
(Client.builder()
    .proxy("http://proxy.corp:3128", scheme="https")   # "all" | "http" | "https"
    .timeouts(connect_seconds=10.0, read_seconds=120.0)
    .gzip(False)
    .tcp_nodelay(False)
    .build())
```

Without an explicit proxy, requests ignore whatever proxy environment the
process was started with — which on a locked-down network looks like an
unexplained connect timeout.

`sanitize_json_schema(schema, dialect)` applies a provider's
constrained-decoding normalization without making a request, for seeing
what will actually be enforced before a rejection tells you.

## Resolver callbacks

For credentials or routing that cannot be stated up front:

```python
(Client.builder()
    .auth_resolver(lambda iden: AuthData.key(cache[iden.adapter_kind.name]))
    .model_mapper(lambda iden: ALIASES.get(iden.model_name, iden.model_name))
    .service_target_resolver(lambda target: reroute(target))
    .build())
```

Callbacks must be regular `def`s. They are called synchronously while
holding the GIL, so an `async def` would never be awaited — it is rejected
at registration — and blocking inside one stalls every other Python thread.
Cache in Python rather than doing I/O in a callback.

## Development

```bash
maturin develop          # build the extension into the active venv
python -m pytest         # Python tests (no network; a local fake provider)

# Rust tests link against libpython, so they need the extension-module
# feature off and the interpreter feature on:
cargo test --no-default-features --features test-interpreter
```

`cargo build` does not link the cdylib on macOS (it lacks maturin's link
arguments); use `cargo check` or `maturin develop`.

## Migrating

See [MIGRATION.md](MIGRATION.md) for the move from the `Client.with_*`
constructors to `ClientBuilder`.

