Metadata-Version: 2.5
Name: decidr
Version: 0.2.0
Summary: Typed decisions from local LLMs in one forward pass, via Ollama
Project-URL: Homepage, https://github.com/devanmolsharma/decidr
Project-URL: Issues, https://github.com/devanmolsharma/decidr/issues
License: MIT
License-File: LICENSE
Keywords: classification,decisions,llm,logprobs,ollama,routing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# decidr

**Typed decisions from local LLMs, in one forward pass.**

Most decisions an application asks an LLM to make are small: *route this ticket*, *is this evidence sufficient*, *how angry is this customer*. A chat model can answer them, but it generates a sentence, or JSON, which your code then parses back into an `if` statement.

`decidr` skips that. It gives the model your options, runs **one forward pass**, and reads the probability of each option directly out of the model's own logits. No answer sentence. No JSON to repair. No decoding loop.

```python
from decidr import Client

client = Client(model="qwen3.5:4b")

decision = client.decide({
    "id": "route-1",
    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
    "question": "Which queue should handle this request?",
    "options": [
        {"id": "access",  "description": "Account access and authentication support."},
        {"id": "billing", "description": "Billing and payment support."},
        {"id": "sales",   "description": "Sales and product evaluation."},
    ],
})

decision.choice          # 'access'
decision.confidence      # 0.9999
decision.probabilities   # {'access': 0.9999, 'billing': 0.0001, 'sales': 0.0}
decision.is_reliable()   # True
```

Works with any model you already have in [Ollama](https://ollama.com). No fine-tuning, no extra runtime, no separate model to download.

## Install

```bash
pip install decidr
```

Requires Python 3.10+ and a running Ollama. Zero dependencies — it's stdlib `urllib` and `math`.

## How it works

Three things, in order:

**1. Options become letters.** Each option is presented to the model as `A`, `B`, `C`… rather than by its own name. This is not cosmetic: real labels like `billing` or `SYS_OUTAGE` are usually *several* tokens, and you cannot read a single-token probability for a multi-token string. Letters are single tokens in every vocabulary. The meaning lives in the descriptions, which is where the model actually reads it.

**2. One forward pass, no generation.** The prompt ends where the answer begins, and generation is capped at a single token. Reasoning mode is explicitly disabled — on a hybrid-reasoning model, a `<think>` preamble would put thinking tokens in the answer slot, and the next token would stop being the decision.

**3. The scores are read, not sampled.** Instead of taking whichever letter the model emitted, `decidr` reads the log probability of *every* option letter and normalizes over them. You get a distribution, not just a pick — so you can threshold on confidence, route ambiguous cases to a human, or log calibration over time.

## Two modes, picked automatically

How completely `decidr` can read those scores depends on your Ollama build. It probes once per process and tells you which mode you're in via `decision.mode`.

| Mode | When | Behavior |
|---|---|---|
| `ranked` | **Stock Ollama** (what you have today) | Falls back to `top_logprobs` (capped at 20 by the API). Options whose letters don't surface in that window are reported in `decision.unscored`. |
| `exact` | Ollama with [`logprob_tokens`](https://github.com/ollama/ollama/pull/18580) | Every option's probability is read directly from the full distribution, regardless of rank. |

**Why this distinction exists:** stock Ollama can only tell you about tokens that rank in the model's top 20 guesses. For a 3-option decision, the letters almost always make that cut and `ranked` is fine. For a 12-option decision, they often don't:

```
12 options, same model, same prompt:
  exact    ->  scored 12/12   reliable=True
  ranked   ->  scored 11/12   reliable=False   unscored: ['cat9']
```

`decidr` never invents a number for an option it couldn't measure. It reports it as unscored, `is_reliable()` returns `False`, and the remaining probabilities are normalized over what was actually observed. A fabricated floor value would be indistinguishable from a real measurement, which is the one thing a probability API must never do.

### Getting `exact` mode

`logprob_tokens` is a change I've proposed upstream to Ollama — **it is not merged yet**:

- Issue: [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579)
- PR: [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580)

Until it lands (if it lands), `decidr` works today in `ranked` mode against stock Ollama. Nothing here depends on that PR being accepted — `exact` mode is an upgrade, not a requirement. If you want it now, you can build from [the branch](https://github.com/devanmolsharma/ollama/tree/classification-logprobs).

## API

### `Client(model, host=..., timeout=..., temperature=..., force_mode=None)`

- `model` — any Ollama model name, e.g. `"qwen3.5:4b"`.
- `host` — defaults to `http://127.0.0.1:11434`.
- `temperature` — applied to the softmax over option logprobs, **not** to sampling. Higher values flatten confidence. Use this to calibrate against a labeled set (see below).
- `force_mode` — skip the capability probe and pin `"exact"` or `"ranked"`.

### `client.decide(row) -> Decision`

`row` needs `id`, `state`, `question`, and 2–16 `options`, each with an `id` and a `description`. `state` may be a string, dict, or list. Malformed rows raise `DecisionError` naming the problem rather than silently producing a confident wrong answer.

### `Decision`

| Field | |
|---|---|
| `choice` | option id with the highest probability |
| `confidence` | probability of `choice` |
| `probabilities` | option id → probability, sums to 1 **over scored options** |
| `logprobs` | raw log probabilities before normalizing |
| `mode` | `"exact"` or `"ranked"` |
| `unscored` | options the server could not report |
| `is_reliable()` | `False` when anything went unscored |
| `raw_answer` | the letter the model actually emitted |

`client.decide_all(rows)` runs a list sequentially.

## Calibration

Probabilities are conditional on the option set you supplied and are not calibrated by default. `choice` is usually right. `confidence` on its own is not a true probability until you check it against real outcomes.

`decidr.calibrate` fits one temperature to your own labeled data and rescales the probabilities:

```python
from decidr import Client
from decidr.calibrate import fit_temperature, evaluate_out_of_fold

client = Client(model="qwen3.5:4b")
labeled = [(client.decide(row), row["correct_id"]) for row in labeled_rows]

fit_temperature(labeled)
# CalibrationResult(temperature=1.8, n=200, ece_before=0.15, ece_after=0.06, accuracy=0.83)

evaluate_out_of_fold(labeled, folds=5)
# fits T on 4 folds, scores it on the 5th, repeated for every fold.
# use this number, not fit_temperature's, since a temperature can overfit
# to the exact sample it was fit on.
```

Rescaling by a constant can't change which option wins, so `choice` stays fixed and only `confidence` moves. Without labeled data, `confidence` still ranks options correctly relative to each other, just treat the absolute number as approximate.

## What this is not

- **Not a new model.** It's a way of reading models you already run.
- **Not a fine-tune.** Nothing is trained; taxonomies change per request.
- **Not novel.** Reading option logits in a single pass is an established technique — [TypeSafe's Jev](https://www.typesafe.ai), [Laya](https://huggingface.co/convaiinnovations/laya), and [SemIf](https://github.com/TheoLeeCJ/SemIf) all do versions of it, and SemIf in particular has done far more rigorous benchmarking and calibration work than this has. `decidr`'s only claim is a narrow one: it's the smallest possible version that runs against an Ollama you already have, with zero dependencies and no model downloads.

## Limitations

- Sequential — no batching yet. One decision, one request.
- 2–16 options (letters `A`–`P`).
- `ranked` mode's completeness degrades as option count grows.
- Probabilities uncalibrated by default; fixable with `decidr.calibrate` and your own labeled data (see above).
- Tested against `qwen3.5:4b`. Small models (<1B) often won't treat a bare letter as a plausible next token; `decidr` raises a clear error rather than returning noise if none of the letters are scored.

## License

MIT
