Metadata-Version: 2.4
Name: wassitai
Version: 0.3.0
Summary: Universal structured-generation & LLM-output parsing framework
Author-email: Abdelaziz Kella <kaa.kella.abdelaziz@gmail.com>, Abdelaziz Kella <a.kella@univ-chlef.dz>
License: MIT
Project-URL: Homepage, https://github.com/kaaaziz/wassitai
Project-URL: Repository, https://github.com/kaaaziz/wassitai
Project-URL: Issues, https://github.com/kaaaziz/wassitai/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.11
Requires-Dist: httpx>=0.24
Requires-Dist: json_repair>=0.25
Requires-Dist: xmltodict>=0.13
Provides-Extra: api
Requires-Dist: fastapi>=0.110; extra == "api"
Requires-Dist: uvicorn[standard]>=0.29; extra == "api"
Requires-Dist: python-multipart>=0.0.9; extra == "api"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-timeout; extra == "dev"
Requires-Dist: hypothesis>=6; extra == "dev"
Requires-Dist: respx>=0.21; extra == "dev"
Requires-Dist: ruff<0.17,>=0.16; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

# Wassitai

Ask an LLM a question, get back a **validated Python object** — not a string you have to parse.

Wassitai sends your prompt with a schema, picks the best structured-output method the provider
supports, then extracts, repairs, validates and coerces the reply through a deterministic pipeline.
If the result still doesn't fit the schema, it re-asks the model. Ships as the importable package
`wassitai`, with a REST API and a browser playground on top.

```bash
pip install wassitai            # library
pip install "wassitai[api]"     # + REST API and playground
```

Code written against the pre-0.2 name still works: `llm_parser` remains importable as a deprecated
alias for `wassitai` until 2.0. If you have an old editable install of the `llm_parser` *distribution*,
run `pip uninstall llm_parser` first — pip won't replace it, and its source tree would keep shadowing
the alias.

## Features

- 🤖 **Any OpenAI-compatible provider** — Groq, OpenAI, DeepSeek and OpenRouter as presets;
  Together, xAI, Ollama, vLLM or a local server with just `base_url=` and `model=`
- 🧱 **Five ways to write a schema** — Pydantic model, `{"field": "type"}` dict, JSON Schema, or a
  JSON or XML schema string
- 🔁 **Object, JSON or XML — in and out** — parse either format the model answers in, render the
  validated result as either
- 📊 **Lists** — ask for `list[Invoice]`, get a list back
- 🧪 **Repairs before it fails** — lifts JSON out of markdown fences and prose, fixes trailing
  commas, smart quotes and Python-repr output, then re-asks the model if it still doesn't fit
- 🔒 **Strict or flexible validation** — exact types for production, coercion while prototyping
- 💡 **Diagnostics you can act on** — every stage reports what it did, a confidence score falls as
  repairs accumulate, and errors are a typed tree with fixes in the message
- 🎯 **Pydantic all the way through** — your model is the schema, the validator and the return type
- 🌐 **A playground, not just a library** — browser console plus a beginner's guide
- ✅ **1,100+ tests** — unit, integration and end-to-end, no network calls unless you
  opt in to the live provider test (see below)

## Python

```python
from wassitai import Client
from pydantic import BaseModel

class Invoice(BaseModel):
    number: str
    total: float

client = Client.from_provider("groq", api_key="…", model="llama-3.3-70b-versatile")
invoice = client.generate("Extract the invoice from: …", schema=Invoice)   # -> Invoice

# A list, by asking for a list schema:
invoices = client.generate("Extract all invoices: …", schema=list[Invoice])

# Never raises — for batch pipelines:
res = client.try_generate("…", schema=Invoice)     # -> Result[Invoice]
if res.ok:
    print(res.value, res.confidence)

# Text you already have, no provider call:
from wassitai import parse, render
invoice = parse('{"number": "A1", "total": 9.5}', Invoice)
text = render(invoice, "json")
```

A schema can be a Pydantic model, a plain `{"field": "type"}` dict, or a JSON Schema.
`strict=True` enforces exact types; the default coerces close-enough values (`"3"` → `3`).
`max_retries` caps how often the model is re-asked (default 1). One dict form is the exception: a
dict written as a real JSON Schema (a `properties`/`type` object) only checks that required keys
are present — declared field types and `strict` are not enforced, since doing that honestly needs
a JSON-Schema-to-Pydantic converter this library doesn't have yet. The `{"field": "type"}` and
`{"model": ..., "attributes": {...}}` dict forms, and a JSON/XML schema string, are all
type-checked like a Pydantic model.

### When the model answers in XML

Nothing asks for XML — structured-output APIs speak JSON — but models ignore instructions, and
stored answers get replayed through `parse()`. An XML body is read into the same validated object a
JSON one produces:

```python
parse("<person><name>Ada</name><age>36</age></person>", Person)   # Person(name='Ada', age=36)
parse('<person name="Ada" age="36"/>', Person)                    # the same
parse("<ns:person><ns:name>Ada</ns:name></ns:person>", Person)    # and the same
```

The format is detected from the answer's own shape, so a JSON body is never read as XML (nor the
reverse) — including a JSON answer behind a `<think>…</think>` tag, and an XML answer whose text
happens to contain braces.

Four things XML cannot express, and what each one does. Every one of them records a diagnostic, so
the intervention is visible in `result.diagnostics` rather than silent:

| in the answer | what happens |
|---|---|
| `<r id="7">` — an attribute | reaches the `id` field, **unless** a sibling `<id>` is also present: then neither is renamed, both survive, and the collision is a warning |
| `<ns:name>` — a namespace prefix | same rule, same reason: stripped when the local name is free, kept when it would collide |
| `<tag>x</tag>` once, for a `list[str]` field | wrapped into `["x"]` — the format renders a list of one exactly like a scalar, and the schema is the only thing that can tell them apart |
| `<name/>` — an empty element | stays `None`: `Optional[str]` accepts it, a required `str` rejects it by name. It is not turned into `""`, which would be a value the model never sent |
| `<r>text<child/>more</r>` — mixed content | a `RepairError`. There is no reliable mapping to a flat schema, and any rule invented for it would produce plausible, silently wrong data |

**`strict=True` cannot enforce scalar types on an XML body**, because the format does not carry
them: every leaf `xmltodict` returns is a string, so `<age>36</age>` and `<age>"36"</age>` are the
same document. Declared `int`/`float`/`bool` fields are therefore converted before validation in
both modes, and the conversion is recorded as a diagnostic. Strict mode still means exactly what it
meant for a JSON body — *the model sent the wrong type* — which is a claim XML has no way to make.
A value that is genuinely not a number (`<age>twelve</age>`) is left alone and still fails.

### Capturing failed runs

Turn on capture to write failing and repaired runs to disk, so the pipeline can be improved from evidence rather than memory.

```python
from wassitai import Capture, Client, JsonlSink

with Capture(
    sink=JsonlSink("./captures"),
    experiment="tool-calling-vs-prompt-repair",
) as capture:
    client = Client.from_provider("groq", api_key="…", capture=capture)
    client.try_generate("List the invoice lines", schema=Invoice)

print(capture.stats())
```

Two files appear in `./captures` per process: a `.jsonl` of records and a `.manifest.json` holding
counters. The manifest's `counters.runs` counts every run, including the clean ones — the denominator
that makes two experiments comparable.

By default, runs that fail **and** runs that needed repair are captured; pass `when="failures"` or
`when="all"` to change that. (Pass `when="failures_and_repairs"` explicitly to document the chosen
policy in the manifest.) The `repaired` outcome covers two different mechanisms — a fix applied inside
the pipeline, where the record's `result.attempts` is 1, and a re-ask of the model, where it is more
than 1 — so `repaired` on its own does not mean "retried"; read `attempts` to tell the two apart.

**What is stored.** The prompt, the schema, every attempt's raw answer, the parsed result, the error
detail, the provider and model names, strategy, config, token usage, and timestamps. Records are
written unredacted, so treat them as sensitive — the parsed result in particular carries the model's
structured output. Diagnostic messages are redacted too — they can carry the model's output verbatim,
so the redactor is applied to prompts, answers, result, error fields and diagnostic text alike.
Three fields are exempt from *redaction*: `request.schema`, `request.config` and `metadata` are
stored **unredacted**, so a schema supplied as a plain dict — whose values you chose — is written
exactly as you passed it. All three are values you supplied rather than anything the model produced,
which is why the redactor is not run over them; the API-key scrubber below still is, and so is the
`max_field_bytes` cap — an exemption from redaction is not an exemption from being bounded, and
`schema` is caller-supplied over HTTP. A field shortened that way is named in `truncated`, as always.

```python
Capture(sink=JsonlSink("./captures"), experiment="e",
        redact=lambda text: EMAIL_RE.sub("[email]", text),
        max_field_bytes=16_384)
```

Text fields are capped at `max_field_bytes` (default 65536, `None` for unlimited), and any field that
was shortened is listed in the record's `truncated` array — a record never silently misrepresents
what the model said. No API key is ever taken from the request side: a record is assembled from the
schema, the model's answer and the result, and credentials are in scope of none of them. The response
side needs more than that, because a provider's error body reaches the record through the exception
message (`wassitai/providers/transport.py` puts up to 300 characters of it into `AuthenticationError`
and `RateLimitError`), and some providers echo the submitted key in a 401 body. So the key you
configured is removed from every string in the finished record, automatically and before your own
`redact` callable ever runs — including the three fields exempt from redaction above. You do not need
to write a redactor for it.

If the sink fails — a full disk, a read-only volume — capture warns once, disables itself, and your
run continues. A problem with one record is handled differently: a broken `redact` callable, or a
value that cannot be serialised at all, drops that record, counts it in `dropped`, and warns once,
but capture continues. Pass `strict_sink=True` to make either raise instead.

## REST API & playground

```bash
uvicorn wassitai.api.app:app --reload
```

Playground at `http://localhost:8000/`, beginner's guide at `/static/guide.html`,
OpenAPI reference at `/docs`.

| Method | Path | Does |
|---|---|---|
| `POST` | `/v1/generate` | prompt + schema → validated object |
| `POST` | `/v1/parse` | text + schema → validated object (no provider call) |
| `GET` | `/v1/providers` | providers and their capabilities |
| `GET` | `/health` | liveness |

```bash
curl -X POST http://localhost:8000/v1/generate \
  -H "X-Provider-Key: $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"provider":"groq","prompt":"Extract the invoice: …",
       "schema":{"number":"str","total":"float"}}'
```

Provider keys travel per-request in the `X-Provider-Key` header (or come from `<PROVIDER>_API_KEY`
on the server) and are never stored, logged, or pre-filled into the browser. Errors map to status
codes: `422` validation · `401` auth · `429` rate limit · `502` provider · `504` timeout.

The server bounds request size. A body over `WASSITAI_MAX_BODY_BYTES` (1 MiB by default) is
refused with `413` before it's read into memory; a `text` field over `WASSITAI_MAX_TEXT_CHARS`
(262144 characters by default) on `/v1/parse` is refused with `422`, naming the field. Set either
to `0` to disable it. Both are backstops — the shipped `Dockerfile` runs uvicorn directly and
`render.yaml` puts nothing in front of it — not a substitute for a limit at your own proxy.
`/v1/generate`'s `prompt` has no character cap of its own — only `WASSITAI_MAX_BODY_BYTES` bounds
it — since it costs provider tokens rather than parse work, and an operator who configures a
server-side `*_API_KEY` has already taken on that spend deliberately.

### Capture

`/v1/generate` and `/v1/parse` both accept three optional fields:

| Field | Default | Meaning |
|---|---|---|
| `include_record` | `false` | Return this run's capture record in the response |
| `capture_consent` | `false` | Allow the server to store this run |
| `experiment` | `null` | A label grouping runs for later comparison |

`run_id` comes back whenever a run was attempted — success or failure — whether or not `include_record`
was set. It's the handle to quote when asking for a specific run to be deleted, the only workable
deletion story for a record that otherwise carries no identity. A request rejected *before* a run
begins has no run to identify, and so no `run_id` either — no API key at all (`401`), an unknown
provider or a strategy/model combination that can't be built, and a schema that fails to resolve all
reject the request before anything is attempted.

A key that *was* sent and the provider refused is the other side of that line, and answers `401`
too: that request was built, dispatched and answered, so it is a run like any other and **does**
come back with a `run_id`. The distinction is what the id is for — there is a run to point at only
in the second case.

```bash
curl -X POST http://localhost:8000/v1/parse \
  -H "Content-Type: application/json" \
  -d '{"text":"{\"number\":\"A1\",\"total\":9.5}",
       "schema":{"number":"str","total":"float"},
       "include_record":true}'
```

The server **stores** a run only when both `capture_consent` is true *and* the
`WASSITAI_CAPTURE_DIR` environment variable is set on the server — either alone stores nothing.
Stored records are swept for retention at API startup and are deleted after 30 days.

Whether that configuration actually *works* is a separate question from whether it's set: a
directory can be named and still not be writable by the server process — wrong ownership on a
mounted disk, a full disk, a permission revoked while the process is running — in which case
capture construction or a later write fails, and the server keeps serving normally without
storing anything and without saying so on its own. `GET /health` reports the real answer as
`storage_active`: `true` only once a persisting capture has actually opened successfully for
this process and hasn't since failed a write; `false` otherwise. It is never inferred from
`WASSITAI_CAPTURE_DIR` being set alone. The hosted playground's consent checkbox derives its
copy from this field (`wassitai/api/static/app.js`) instead of asserting a fixed promise, so a
storage failure degrades to an honest "not storing here" — or, if the check itself couldn't be
reached, "couldn't be confirmed" — rather than a notice the deployment can't back up.

Records are grouped by `experiment` label, each with its own writer, and a bare `storage_active`
aggregates across every label already open in the process — which can disagree with what one
*specific* label will actually experience: an unrelated label's sink failure makes the aggregate say
`false` for a label that is itself fine, and a brand-new label opened after the process has already
seen many distinct ones can be silently declined (a fixed per-process cap on distinct labels, sized
generously for legitimate use) while the aggregate, having never heard of it, still says `true`.
`GET /health?experiment=<label>` answers for that specific label instead, and is what the playground
actually asks — with its own experiment field, including when it's empty.

**On the hosted playground**, `render.yaml` sets `WASSITAI_CAPTURE_DIR` to a mounted persistent
disk, so a consented run survives a redeploy and is purged 30 days later. **Self-hosting** stores
nothing until you set `WASSITAI_CAPTURE_DIR` yourself — and that path needs to sit on storage that
survives a restart, or every record is lost the moment the container does. Most platforms only
offer that on a paid tier and typically cap such a volume to a single attached instance, which
rules out running more than one replica; on Render specifically, both are confirmed in
[their disk docs](https://render.com/docs/disks). To have a specific run deleted before its 30 days are up,
quote its `run_id` through [Contact](#contact) below.

`include_record` **returns** the record if one was produced. When storage isn't active — no consent,
or consent without `WASSITAI_CAPTURE_DIR` set — that's every outcome, the same as the no-consent case.
When storage *is* active (both conditions above), the server persists under the library's default
policy, `failures_and_repairs` — so a clean run that was consented **and** stored still comes back
with `record: null`, honestly, because nothing was recorded; `run_id` still comes back regardless.

**What is stored.** The prompt, the schema, every attempt's raw answer, the parsed result value, the
error detail, the provider and model names, the strategy, the config, diagnostics, token usage and
timestamps — the same fields Slice 1 stores locally (see "Capturing failed runs" above). Never the
API key, the request's IP address, or its headers.

The API caps text fields at 32 KB — half the library default (`65536`), because a record now travels
over the wire. That cap is **per field**, not per response: a record with many attempts can still add
up to more than 32 KB.

Two shapes a consumer should expect:

- A `result.value` that exceeded the cap is stored as a **truncated string**, not an object or a
  list — check its type before assuming it parses.
- `notes` names any field that had no native JSON form and was stored as a `repr()` instead, and any
  field whose value degraded — for example `request.model`, when the provider in use exposes no model
  name.

## How it works

`generate()` → schema adapter → strategy → provider → **pipeline** → validated object.

- **Strategy** — resolved per provider in order `native` (JSON-schema mode) → `tool_calling` →
  `prompt_repair`, which is the universal floor and needs nothing from the provider.
- **Pipeline** — six ordered stages: `normalize → extract → syntax_repair → deserialize →
  validate → coerce`. Each records a diagnostic, and the confidence score falls as repairs and
  re-asks accumulate.
- **Extending** — `Provider`, `Strategy`, `SchemaAdapter` and `Stage` are the four interfaces;
  most providers need only a preset in `wassitai/providers/__init__.py`.

Errors form one tree under `LLMParserError`: `ConfigurationError` (your bug), `ProviderError`
(the network/provider's), and `ParseError` (the model's).

## Providers

Built-in presets speak the OpenAI wire format: **groq**, **openai**, **deepseek**, **openrouter**.
Any other OpenAI-compatible endpoint (Together, xAI, Ollama, vLLM, local) works by passing
`base_url=` and `model=`. Native adapters for Anthropic, Gemini and Bedrock are planned behind the
same `Provider` interface.

## Why Wassitai

| | Wassitai | Typical structured-output library |
|---|---|---|
| **Providers** | 4 presets, plus any OpenAI-compatible endpoint via `base_url=` | An adapter to write per provider |
| **Input** | A JSON *or* XML answer, detected from its own shape | JSON only |
| **Output** | A validated object, or the same data as JSON or XML text | JSON only |
| **Validation** | Strict *and* flexible, chosen per call | One fixed mode |
| **On a bad answer** | Repairs it, then re-asks the model | Raises and stops |
| **When it fails** | Per-stage diagnostics and a confidence score | A single validation error |
| **Ships a UI** | Playground + beginner's guide | Library only |

### Key differentiators

1. **Dual validation modes** — coerce close-enough values while prototyping, enforce exact types in
   production, without changing your schema.
2. **It repairs before it gives up** — malformed JSON is fixed, and a reply that still doesn't fit
   the schema is sent back to the model rather than thrown at you.
3. **It tells the truth about what happened** — every stage records a diagnostic, and confidence
   drops as repairs and re-asks accumulate, so a "valid" object you shouldn't trust is visible.
4. **Any OpenAI-compatible endpoint** — Together, xAI, Ollama, vLLM or a local server need a URL
   and a model name, not an adapter.
5. **JSON *and* XML** — an answer in either format parses to the same validated object, which
   renders back to either.
6. **A playground, not just a library** — a browser console that exercises the real REST API, with
   a guide written for people who have never called an LLM.

### Use cases

- **Development** — flexible validation for rapid prototyping
- **Production** — strict validation for data integrity
- **Research** — compare providers and strategies on the same prompt and schema
- **Education** — plain-language guide and honest error messages
- **Experimentation** — the playground, with no code to write

## From a checkout

```bash
pip install -e ".[api,dev]"              # editable install with the API and test deps
pytest                                   # 1,100+ tests, no network calls unless you set
                                          # WASSITAI_LIVE and GROQ_API_KEY (opt-in, see tests/bench/)
pytest -m unit                           # or: integration, e2e, provider
pytest --cov=wassitai --cov-report=html
```

CI runs the same suite on Python 3.9–3.13, plus `ruff` over the current engine, the REST API and the
`llm_parser` alias (the pre-0.2 `core/` and `llms/` modules are grandfathered).

## Legacy API (removed in 2.0)

`LLMParser.ask` still works and still supports Gemini and HuggingFace, but is deprecated.

| Old | New |
|---|---|
| `LLMParser("groq", api_key=k).ask(q, schema=S, fmt="json")` | `Client.from_provider("groq", api_key=k).generate(q, schema=S)` |
| `...ask(q, schema=S, many=True)` | `...generate(q, schema=list[S])` |
| `strict_validation=True` | `generate(..., strict=True)` |
| need the JSON string back | `render(obj, "json")` |

The Streamlit UI at `ui/app.py` (`streamlit run ui/app.py`) targets this legacy path; the
playground above is the current one.

## License

MIT.

## Contact

Issues and discussions on [GitHub](https://github.com/kaaaziz/wassitai) ·
kaa.kella.abdelaziz@gmail.com
