Metadata-Version: 2.5
Name: pydecide
Version: 0.1.3
Summary: One client for every decision model: Choice, Score and Noul over Jev, OpenRouter, laya, MLX, CrossEncoders and an LLM fallback.
Project-URL: Homepage, https://github.com/gopaljigaur/decide
Project-URL: Issues, https://github.com/gopaljigaur/decide/issues
Author: Gopalji Gaur
License-Expression: MIT
License-File: LICENSE
Keywords: classification,decision model,jev,laya,routing,system one,typesafe
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.110
Requires-Dist: httpx>=0.27
Requires-Dist: uvicorn>=0.29
Provides-Extra: all
Requires-Dist: laya-mlx>=0.1; (python_version >= '3.11') and extra == 'all'
Requires-Dist: laya>=0.3; extra == 'all'
Requires-Dist: sentence-transformers>=3.0; extra == 'all'
Requires-Dist: torch; extra == 'all'
Provides-Extra: laya
Requires-Dist: laya>=0.3; extra == 'laya'
Provides-Extra: mlx
Requires-Dist: laya-mlx>=0.1; (python_version >= '3.11') and extra == 'mlx'
Provides-Extra: server
Provides-Extra: st
Requires-Dist: sentence-transformers>=3.0; extra == 'st'
Requires-Dist: torch; extra == 'st'
Description-Content-Type: text/markdown

# pydecide

[![CI](https://github.com/gopaljigaur/decide/actions/workflows/ci.yml/badge.svg)](https://github.com/gopaljigaur/decide/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/pydecide)](https://pypi.org/project/pydecide/)

`decide` is one Python client for typed decisions - Choice, Score and Noul -
over any "System One" decision model: TypeSafe's hosted Jev, OpenRouter's
Decisions endpoint, the open-weight laya family (PyTorch and MLX), any
sentence-transformers CrossEncoder, and a JSON-prompted LLM fallback. A
`Client` takes an ordered list of backends and a confidence policy: if a
backend errors or answers with low confidence, the next backend is tried.
The library also ships a small HTTP server that speaks TypeSafe's wire
protocol, so existing TypeSafe clients can point at a local model instead.

## Install

```bash
uv tool install pydecide   # the decide CLI on your PATH, from any directory
uv add pydecide            # inside a project
```

With pip: `pip install pydecide` / `pipx install pydecide`. Either way, this
includes the CLI, the hosted backends and the server.

Local model backends need an extra:

```bash
uv tool install "pydecide[all]"
```

With pip: `pip install "pydecide[all]"`. Quote the brackets in zsh, since it
otherwise tries to glob them. This pulls in PyTorch and MLX, about 2 GB.

| Extra | Adds |
|---|---|
| `server` | Nothing further; kept as an empty extra so `pip install "pydecide[server]"` still works. FastAPI and uvicorn ship in the base install. |
| `laya` | `laya` backend (PyTorch) |
| `mlx` | `laya_mlx` backend (Apple Silicon; needs Python 3.11+) |
| `st` | `crossencoder` backend (sentence-transformers) |
| `all` | `laya` + `mlx` + `st` |

The `mlx` extra depends on `laya-mlx`, which requires Python 3.11 or newer;
on 3.10 the extra installs nothing and the `laya_mlx` backend is
unavailable.

## Quickstart: zero configuration

Install a local backend and `decide ask` works right away, with no
environment variables at all:

```bash
uv tool install "pydecide[mlx]"    # Apple Silicon
# or
uv tool install "pydecide[laya]"   # any platform (PyTorch)
```

```bash
decide ask "I had a rough day, everything broke" --noul "Is the writer doing well?" --choice good,bad
```

```
NAME    TYPE    ANSWER  PROBABILITIES
choice  choice  bad     good=0.01 bad=0.99
noul    noul    false   0.01
backend=laya_mlx route=laya_mlx:ok latency=33.1ms
```

This is the real output of that command, run with `laya_mlx` installed and
none of `DECIDE_LOCAL_MODEL`, `TYPESAFE_API_KEY`, `OPENROUTER_API_KEY`,
`OPENAI_API_KEY`, `DECIDE_LLM_BASE_URL` or `DECIDE_BACKENDS` set.

The Python client works the same way. `Client.from_env()` builds a backend
chain from whatever is installed and configured in the process environment;
with a local backend installed, it needs no variables either. Pass an
explicit `env=` mapping instead (e.g. in tests) to configure from something
other than `os.environ`:

```python
from decide import Client, Choice, Score, Noul

client = Client.from_env()

r = client.decide(
    state={"ticket": "I was charged twice, please refund."},
    questions={
        "team": Choice(
            "Which team handles this?",
            {
                "billing": "charges, refunds",
                "engineering": "bugs and outages",
                "sales": "pricing and upgrades",
            },
        ),
        "severity": Score("How severe is this?", ["minor", "degraded", "blocked"]),
        "refund": Noul("Does the customer ask for money back?"),
    },
)

r.choices["team"].choice  # "billing"
r.choices["team"].probabilities  # {"billing": 1.0, "engineering": 0.0, "sales": 0.0}
r.scores["severity"].score  # 1.5522  (expected level index, 0..len(levels)-1)
r.scores["severity"].probabilities  # [0.0197, 0.4083, 0.5719]
r.nouls["refund"].noul  # 0.9461
r.meta.backend  # "laya_mlx"
r.meta.latency_ms  # 37.1
r.meta.route  # ["laya_mlx:ok"]
```

This was run against the `laya_mlx` backend with no environment variables
set at all, using its own default checkpoint; every value above is the real
output of that run, not illustrative.

`from_env` picks the chain in this order, using whatever is installed and/or
configured: `laya_mlx` or `laya`, whichever is importable, with its own
default model unless `DECIDE_LOCAL_MODEL` overrides it; `typesafe` (if
`TYPESAFE_API_KEY` is set); `openrouter` (if `OPENROUTER_API_KEY` is set);
`llm` (if `DECIDE_LLM_BASE_URL` or `OPENAI_API_KEY` is set). A local backend,
when installed, is tried first, with the hosted backends as fallback. Set
`DECIDE_BACKENDS="laya,typesafe"` to override the order explicitly. If
nothing is importable or configured, `from_env` raises `ConfigError` telling
you to install a local backend or set a hosted one's API key.

`AsyncClient` has the same surface, `await`ed: `await client.decide(...)`,
`await client.decide_batch(...)`, `AsyncClient.from_env(...)`.

## Backends

| Module | Backend name | Extra | Notes |
|---|---|---|---|
| `typesafe.py` | `typesafe` | none (httpx only) | `POST {base_url}/v1/systemone`, bearer auth. Default base URL `https://api.typesafe.ai`, default model `jev-latest`. |
| `openrouter.py` | `openrouter` | none | Same wire shape as `typesafe`, `POST https://openrouter.ai/api/alpha/decisions`, default model `typesafe/jev-latest`. |
| `laya.py` | `laya` | `pydecide[laya]` | Local PyTorch `laya.Agent`, default model `convaiinnovations/laya` when constructed directly. |
| `laya_mlx.py` | `laya_mlx` | `pydecide[mlx]` (Python 3.11+) | Local MLX `laya_mlx.Agent` (Apple Silicon), default model `aac6fef/laya-multilingual-mlx` when constructed directly. |
| `crossencoder.py` | `crossencoder` | `pydecide[st]` | Local `sentence_transformers.CrossEncoder`. Not configurable from the environment; construct it directly and pass it to `Client([...])`. |
| `llm.py` | `llm` | none (httpx only) | Any OpenAI-compatible chat-completions server, default base URL `https://api.openai.com/v1`, default model `gpt-4o-mini`. The least trustworthy backend (see below). |

`crossencoder` is built by hand, for example:

```python
from decide import Client, Choice
from decide.backends.crossencoder import CrossEncoderBackend

backend = CrossEncoderBackend("cross-encoder/ms-marco-MiniLM-L6-v2")
client = Client([backend])

r = client.decide(
    "I was charged twice for the same order last week, please refund the duplicate charge.",
    {
        "team": Choice(
            "Which team should handle this?",
            {
                "billing": "Charges, invoices, payment problems, refunds",
                "eng": "Bugs, crashes, broken features",
                "shipping": "Delivery status, delays, lost packages",
            },
        )
    },
)
r.choices["team"].choice  # "billing"
r.choices["team"].probabilities  # {"billing": 0.947, "eng": 0.019, "shipping": 0.034}
```

### Environment variables: overrides and hosted-backend keys

None of these are required to get started - see
[Quickstart: zero configuration](#quickstart-zero-configuration) above. They
either override a local backend that `from_env` already auto-selects once
it's installed, or supply the API key a hosted backend needs to be
auto-selected at all.

| Variable | Backend | Meaning |
|---|---|---|
| `DECIDE_LOCAL_MODEL` | `laya`, `laya_mlx` | Overrides the default Hugging Face repo id (or local path) of the model to load. Optional: omit it and the installed local backend auto-selects with its own default model. |
| `TYPESAFE_API_KEY` | `typesafe` | API key, sent as `Authorization: Bearer`. Required to auto-select `typesafe`. |
| `TYPESAFE_BASE_URL` | `typesafe` | Overrides the default `https://api.typesafe.ai`. |
| `OPENROUTER_API_KEY` | `openrouter` | API key, sent as `Authorization: Bearer`. Required to auto-select `openrouter`. |
| `DECIDE_LLM_BASE_URL` | `llm` | Base URL of an OpenAI-compatible chat-completions server. Setting it (or `OPENAI_API_KEY`) auto-selects `llm`. |
| `OPENAI_API_KEY` | `llm` | API key, sent as `Authorization: Bearer`, if the server needs one. |
| `DECIDE_LLM_MODEL` | `llm` | Model name to request; defaults to `gpt-4o-mini`. |
| `DECIDE_BACKENDS` | `Client.from_env` | Comma-separated backend names, overriding auto-detection entirely. |
| `DECIDE_MIN_CONFIDENCE` | `Client.from_env` (`Gate`) | Float threshold for the default `Gate` built by `from_env`, when no explicit `policy` is passed. |
| `DECIDE_API_KEY` | `decide serve` | Bearer token required to call the server, when `--api-key` is not passed. The flag takes precedence over this variable. |

## Gating and fallback

```python
from decide import Gate

Gate(
    min_confidence=0.0,  # top probability of a Choice, max(noul, 1-noul) for a Noul
    per_question=None,  # optional {"question_name": threshold} overrides
    on_error="next",  # or "raise" to stop the chain on the first BackendError
)
```

For each backend in order: call it. On `BackendError` with `on_error="next"`,
append `"<name>:error"` to `meta.route` and try the next backend (with
`on_error="raise"`, the error propagates immediately instead). If the
response fails the gate, append `"<name>:low_confidence:<question>=<value><threshold>"`
and try the next backend. If it passes, append `"<name>:ok"` and return.
Score answers are never gated (there is no single confidence number for an
expected value over levels).

If every backend is exhausted without a passing response, the best response
seen so far (highest minimum confidence across its gated answers) is
returned, with `meta.route[-1] == "<name>:accepted_low_confidence"`. If no
backend produced any response at all, `Client.decide` raises
`AllBackendsFailed(route, errors)`.

Route strings you will see in `meta.route`:

- `"<name>:ok"` - the backend answered and passed the gate.
- `"<name>:error"` - the backend raised a `BackendError`.
- `"<name>:low_confidence:<question>=<value><threshold>"` - the backend
  answered but at least one gated question fell below its threshold.
- `"<name>:accepted_low_confidence"` - appended once, at the end of the
  route, when no backend passed the gate and the best low-confidence
  response was returned instead.

`Client.decide_batch`/`AsyncClient.decide_batch` run the same chain per
input state, preserving input order; a state that clears the gate on an
earlier backend is not sent to later ones. `Client.decide_batch` uses a
backend's real batch path when `capabilities().batch` is true, looping
`decide` per request otherwise. `AsyncClient.decide_batch` always loops
`adecide` per state, concurrently via `asyncio.gather`; it does not use a
backend's batch path in v1. If any state in the batch is left unrouted,
`AllBackendsFailed` carries `partial` (every `Response` that did resolve,
keyed by input index) and `failed` (the route so far for every state that
did not), so the resolved siblings are not silently lost.

## Server: point TypeSafe's SDK at a local model

```bash
DECIDE_LOCAL_MODEL=aac6fef/laya-multilingual-mlx decide serve --backends laya_mlx --port 8811
```

`GET /health` reports liveness and the configured backend names;
`GET /v1/models` lists them in an OpenAI-style shape; `POST /v1/systemone`
takes a TypeSafe `SystemOneRequest` body and returns a
`SystemOneResponse`-shaped body plus a `decide` extension carrying our own
backend/route metadata. Errors come back as
`{"error": {"message": ..., "type": ...}}` with a matching HTTP status.

If `--api-key`/`DECIDE_API_KEY` is set, requests must send a matching
`Authorization: Bearer <token>` header; the key must be ASCII, since HTTP
header bytes are latin-1 decoded by the server before comparison.

A raw request against the running server above:

```bash
curl -s -X POST http://127.0.0.1:8811/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"ticket": "I was charged twice, please refund."},
    "questions": {
      "team": {"type": "choice", "instructions": "Which team handles this?",
               "criteria": {"billing": "charges, refunds", "engineering": "bugs and outages", "sales": "pricing and upgrades"}},
      "severity": {"type": "score", "instructions": "How severe is this?", "criteria": ["minor", "degraded", "blocked"]},
      "refund": {"type": "noul", "instructions": "Does the customer ask for money back?"}
    }
  }'
```

```json
{
  "model": "aac6fef/laya-multilingual-mlx",
  "answers": {
    "team": {"type": "choice", "choice": "billing", "confidence": 1.0,
              "probabilities": {"billing": 1.0, "engineering": 0.0, "sales": 0.0}},
    "severity": {"type": "score", "score": 1.5522, "confidence": 0.5719,
                 "legend": {"0": "minor", "1": "degraded", "2": "blocked"},
                 "probabilities": {"0": 0.0197, "1": 0.4083, "2": 0.5719}},
    "refund": {"type": "noul", "noul": 0.9461}
  },
  "usage": {"input_tokens": 0, "output_tokens": 0},
  "decide": {"backend": "laya_mlx", "latency_ms": 50.5, "route": ["laya_mlx:ok"]}
}
```

And the same request through TypeSafe's own SDK, pointed at the local
server (no API key is needed since this server has none configured, but the
SDK requires a non-empty string):

```python
from typesafe_sdk import TypeSafeClient, Choice, Score, Noul

client = TypeSafeClient(api_key="anything", base_url="http://127.0.0.1:8811")

resp = client.system_one(
    state={"ticket": "I was charged twice, please refund."},
    questions={
        "team": Choice(
            instructions="Which team handles this?",
            criteria={
                "billing": "charges, refunds",
                "engineering": "bugs and outages",
                "sales": "pricing and upgrades",
            },
        ),
        "severity": Score(
            instructions="How severe is this?", criteria=["minor", "degraded", "blocked"]
        ),
        "refund": Noul(instructions="Does the customer ask for money back?"),
    },
)
```

```text
model='aac6fef/laya-multilingual-mlx' usage=Usage(input_tokens=0, output_tokens=0)
answers={'team': ChoiceAnswer(type='choice', choice='billing', confidence=1.0,
  probabilities={'billing': 1.0, 'engineering': 0.0, 'sales': 0.0}),
 'severity': ScoreAnswer(type='score', score=1.5522, confidence=0.5719,
  legend={0: 'minor', 1: 'degraded', 2: 'blocked'},
  probabilities={0: 0.0197, 1: 0.4083, 2: 0.5719}),
 'refund': NoulAnswer(type='noul', noul=0.9461)}
```

TypeSafe's SDK parsed our server's response without modification: it is a
genuine `SystemOneResponse`, not a hand-shaped dict.

## CLI

`--choice`, `--score` and `--noul` question names are optional. An unnamed
flag is auto-named after the flag itself (`choice`, `score`, `noul`); a
second unnamed flag of the same type becomes `choice2`, `score2`, `noul2`,
and so on:

```bash
decide ask "I had a rough day" --noul "Is the writer doing well?" --choice good,bad
```

```text
NAME    TYPE    ANSWER  PROBABILITIES
choice  choice  bad     good=0.07 bad=0.93
noul    noul    false   0.02
backend=laya_mlx route=laya_mlx:ok latency=65.3ms
```

Name a question explicitly with `NAME=...` to control what shows up in the
table and in `--json`; named and unnamed questions can be mixed on the same
command line:

```bash
decide ask "I was charged twice, please refund." \
  --choice "team=billing,engineering,sales" \
  --score "severity=minor,degraded,blocked" \
  --noul "refund=Does the customer ask for money back?"
```

```text
NAME      TYPE    ANSWER    PROBABILITIES
team      choice  billing   billing=0.93 engineering=0.02 sales=0.05
severity  score   degraded  1.09 (degraded)
refund    noul    true      0.87
backend=laya_mlx route=laya_mlx:ok latency=34.8ms
```

`--json` prints a machine-readable payload instead of the table, and writes
only that JSON to stdout, so `decide ask ... --json | jq .` is safe to use
in scripts. `--min-confidence` and `--model` are also available on `ask`.
By default, `ask` and `serve` set `HF_HUB_DISABLE_PROGRESS_BARS=1` (unless
it's already set) so a local backend's model-loading progress bars don't
mix into that output; pass `--verbose` to see them. When shown, that
progress goes to stderr, never stdout.

```bash
decide backends
```

```text
NAME          INSTALLED  CONFIGURED  INSTALL
typesafe      yes        no
openrouter    yes        no
laya          yes        default
laya_mlx      yes        default
crossencoder  yes        n/a
llm           yes        no
```

Every backend with `INSTALLED` `no` gets an `INSTALL` column naming the
exact command to add it, e.g. `pip install "pydecide[laya]"`. For `laya`
and `laya_mlx`, `CONFIGURED` reads `default` when installed with no
`DECIDE_LOCAL_MODEL` override (it will be auto-selected with its own
default model), `yes` when `DECIDE_LOCAL_MODEL` is set, and `no` only when
the backend isn't installed at all.

`decide serve --backends a,b --host 127.0.0.1 --port 8811 [--api-key TOKEN] [--min-confidence FLOAT]`
runs the HTTP server described above.

## What the probabilities mean

Every probability in a `ChoiceAnswer`, `ScoreAnswer` or `NoulAnswer` is
whatever the backend reported; `decide` does not calibrate, smooth or
verify it. What that means differs by backend: `typesafe`, `openrouter` and
`laya`/`laya_mlx` are purpose-built decision models, but their outputs are
still self-reported by the model and not audited by this library. The
`crossencoder` backend turns a relevance reranker's raw logits into a
softmax or sigmoid - the result is a *normalized score* forced to distribute
mass over the supplied candidates, not a calibrated probability; a `Choice`
will still pick a winner even when every candidate is a bad fit, and a
`Noul` of 0.9 does not mean the condition holds 90% of the time. The `llm`
backend is the least trustworthy of all: it prompts a general chat model to
estimate its own confidence in JSON, with no guarantee the model attends to
every candidate or keeps its numbers well calibrated. Treat all of this
accordingly - as a signal to gate and fall back on, not as ground truth.

## Status

`pydecide` is at 0.1.1. The public API may still change before a 1.0
release.

## License

MIT, see [LICENSE](LICENSE).
