Metadata-Version: 2.4
Name: sthai
Version: 1.2.0
Summary: Python client for the SiteHost AI Platform: inference, embeddings and reranking
Keywords: sitehost,ai,llm,inference,embeddings,reranking,vllm
Author: Jordan Russell
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Dist: msgspec>=0.21.1
Requires-Dist: niquests>=3.20.1
Requires-Python: >=3.10
Project-URL: Repository, https://github.com/sitehostnz/sthai-py
Project-URL: Issues, https://github.com/sitehostnz/sthai-py/issues
Project-URL: SiteHost AI Platform, https://kb.sitehost.nz/ai-platform
Description-Content-Type: text/markdown

# sthai-py

[![Tests](https://github.com/sitehostnz/sthai-py/actions/workflows/tests.yml/badge.svg)](https://github.com/sitehostnz/sthai-py/actions/workflows/tests.yml)

A Python client for the [SiteHost AI Platform](https://kb.sitehost.nz/ai-platform): inference, embeddings and reranking with typed requests and responses built on [msgspec](https://jcristharif.com/msgspec/).
Check out our [PHP client port](https://github.com/sitehostnz/sthai-php) as well!

## The SiteHost AI Platform

The [SiteHost AI Platform](https://kb.sitehost.nz/ai-platform) serves capable open-weight models at their full context windows - not heavily quantised cut-downs - with full control over system prompts and outputs. Everything runs on SiteHost's own hardware in their own New Zealand data centres, so your data never leaves the country, and request bodies are never stored: only usage metrics are kept for billing and performance monitoring.

Models are stable targets, too: each served model has a minimum one-year retention window and at least three months' deprecation notice, with guidance on any adjustments needed.

The platform currently serves three models, one per capability (see the [models page](https://kb.sitehost.nz/ai-platform/models) for the source of truth):

| Model | Purpose | Context window |
|-------|---------|----------------|
| `Qwen/Qwen3.6-27B` | Inference (chat, multimodal, thinking) | 262K |
| `Qwen/Qwen3-VL-Embedding-8B` | Embeddings (multimodal, Matryoshka, 4096 dims) | 32K |
| `Qwen/Qwen3-VL-Reranker-8B` | Reranking (multimodal, instruction-trained) | 32K |

## Installation

Requires Python 3.10+. Install from [PyPI](https://pypi.org/project/sthai/):

```bash
pip install sthai
# or, in a uv project
uv add sthai
```

## Getting started

Create an API key in the SiteHost Control Panel (see [API keys](https://kb.sitehost.nz/ai-platform/api-keys)). The client reads it from the `STHAI_KEY` environment variable, or you can pass `api_key=` explicitly:

```python
from sthai import Client

client = Client()  # or Client(api_key="...")
response = client.chat("What's the tallest mountain in New Zealand?")
print(response.output().text)
```

## Usage

### Chat and history

`chat()` keeps a conversation going: each successful call appends the user and assistant turns to the client's history, and later calls send it back. The system prompt is applied per call rather than stored.

```python
client.chat("I'm planning a tramping trip to Fiordland.")
client.chat("What should I pack?")  # the model sees the earlier turn

client.chat("Standalone question.", use_history=False)  # neither sends nor records
client.clear_history()

client.chat("Be brief: why is the sky blue?", system_prompt="You are terse.")
```

The stored conversation is accessible: `history()` returns the turns as role/content dicts, and `set_history()` replaces them, for restoring a persisted conversation. `set_write_history()` toggles recording after construction; turning it off keeps the stored history and still sends it, only recording stops. `history_usage()` sums token usage across the calls that built the stored history. Each call resends the conversation so far, so input tokens count what the server processed (as billed), not unique tokens.

```python
saved = client.history()          # role/content dicts, oldest first
client.set_history(saved)         # restore a persisted conversation
client.set_write_history(False)   # stop recording; stored turns still sent

print(client.history_usage().input_tokens)  # summed across the conversation
```

Thinking models can reason before answering; the reasoning rides along on the response:

```python
response = client.chat("What is 17 * 23?", use_thinking=True, max_tokens=2000)
print(response.output().reasoning)  # or client.last_reasoning()
print(response.output().text)
```

### Images

Chat and embedding inputs can include images, given as URLs or as local files/bytes (PNG, JPEG, GIF or WEBP - files are inlined as data URIs):

```python
from pathlib import Path

client.chat("What's in this image?", image_files=[Path("photo.png")])
client.chat("Compare these.", image_urls=["https://example.com/a.jpg", "https://example.com/b.jpg"])
```

### One-off and structured responses

`response()` mirrors `chat()` without the back-and-forth: the stored history is neither sent nor updated. Pass `response_type=` to get structured output - a msgspec Struct becomes a JSON schema the server enforces during generation, and the decoded, validated instance is returned:

```python
from msgspec import Struct

class CityInfo(Struct):
    name: str
    country: str
    population: int

city = client.response(
    "Give me basic facts about Wellington.",
    response_type=CityInfo,
)
print(city.population)

data = client.response("List three NZ birds as JSON.", response_type=dict)
```

If the output is cut off by the token limit, or (rarely, with thinking enabled) the server skips the schema, parsing raises a `sthai.ResponseParseError` naming the cause.

### Embeddings

`embed()` turns one input - text, images, or both - into a single vector, the sole entry in the response's `output()`. The embedding model is instruction-trained: document embedding is the default, and `query=True` switches to the query instruction for search-style lookups. `dimensions=` truncates the vector server-side (Matryoshka - powers of two work best):

```python
vector = client.embed("The Beehive is New Zealand's parliament building.").output()[0]
query_vector = client.embed("Where does NZ parliament sit?", query=True).output()[0]
small = client.embed("Compact vector, please.", dimensions=512).output()[0]
```

`batch_embed()` embeds many texts in one request; the response's `output()` is one vector per text in order:

```python
vectors = client.batch_embed([
    "Wellington is the capital of New Zealand.",
    "Auckland is the largest city in New Zealand.",
]).output()
```

### Reranking

`rerank()` scores each document against a query; `output()` on the response is the results sorted by relevance, with each result's `index` mapping back to your input list:

```python
response = client.rerank(
    "What is the capital of New Zealand?",
    [
        "The capital of New Zealand is Wellington.",
        "Auckland has the largest population in New Zealand.",
        "The All Blacks are New Zealand's national rugby team.",
    ],
    top_n=2,
)
for result in response.output():
    print(result.relevance_score, result.document.text)
```

Pass `instruction=` to steer relevance for a specific task; the model applies a sensible default otherwise.

### Response helpers

Every inference, embedding and rerank call returns the full response struct, and every response struct has `usage()` (input/output/cached token counts) and `output()` (the useful payload):

```python
response = client.chat("Hello!")
print(response.usage().input_tokens, response.usage().output_tokens)

embedded = client.embed("Hello!")
print(embedded.usage().input_tokens, len(embedded.output()[0]))
```

The one exception is `response(response_type=...)`, which returns the parsed value directly; the full struct from the most recent inference call remains available via `last_response()`:

```python
city = client.response("Describe Wellington.", response_type=CityInfo)
print(client.last_response().usage().input_tokens)
```

### Sessions

Pinning requests to a server session keeps routing consistent and helps caching. Pass `session_pin=` with your own identifier, or let the client generate one:

```python
client = Client(auto_session=True)
print(client.session_pin())  # the pin in use; None when unpinned

client.set_session_pin("my-own-pin")  # pin subsequent requests; None unpins
pin = client.new_session()  # switch to a freshly generated pin
```

### Models and health

```python
client.healthy()  # True if the server is up
for card in client.models():
    print(card.id)
```

### Async

`AsyncClient` mirrors `Client` method-for-method, with every network method awaitable. Like `Client` it is session-less: construct it and go - there is no `async with` or `close()` to manage.

```python
import asyncio
from sthai import AsyncClient

async def main() -> None:
    client = AsyncClient()
    response = await client.chat("What's the tallest mountain in New Zealand?")
    print(response.output().text)

asyncio.run(main())
```

Async is most useful for concurrent fan-out, such as `asyncio.gather` over many `embed()` or `rerank()` calls.

### Error handling

Every error the client raises subclasses `sthai.SthaiError`, so catching it is enough to handle anything sthai can raise - you never need to import niquests or msgspec:

```python
from sthai import APIError, Client, ClientError, InputError, ResponseError, TransportError

client = Client()
try:
    response = client.chat("hello", model="no-such-model")
except ClientError as exc:
    print(exc.status_code, exc.error_type, exc)  # 404 invalid_request_error 404: model not found (invalid_request_error)
except APIError:
    ...  # 5xx: the server had a problem
except TransportError:
    ...  # the request never completed: connection failure, timeout, TLS error
except InputError:
    ...  # bad arguments to a client method
except ResponseError:
    ...  # the server returned something the client couldn't use
```

`ClientError` (4xx) and `APIError` (5xx) both subclass `APIStatusError`, which carries `status_code`, `response`, `server_message` and `error_type` (parsed from the server's error body when present). `ResponseParseError`, raised by `response()`'s structured-output parsing, subclasses `ResponseError`.

## Development

Development uses [uv](https://docs.astral.sh/uv/). The test suite runs entirely offline against fixtures captured from the live API:

```bash
git clone https://github.com/sitehostnz/sthai-py.git
cd sthai-py
uv sync
uv run pytest
uv run ruff check sthai/ tests/
uv run ruff format --check sthai/ tests/
uv run ty check sthai/ tests/
```

To refresh the fixtures against the live API (costs tokens, needs a real key), see `tests/capture_fixtures.py`.

## Licence

[MIT](LICENSE)
