Metadata-Version: 2.5
Name: decidr
Version: 0.1.0
Summary: Typed decisions from local LLMs in one forward pass, via Ollama
Project-URL: Homepage, https://github.com/devanmolsharma/decidr
Project-URL: Issues, https://github.com/devanmolsharma/decidr/issues
License: MIT
License-File: LICENSE
Keywords: classification,decisions,llm,logprobs,ollama,routing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# decidr

**Typed decisions from local LLMs, in one forward pass.**

Most decisions an application asks an LLM to make are small: *route this ticket*, *is this evidence sufficient*, *how angry is this customer*. A chat model can answer them, but it generates a sentence, or JSON, which your code then parses back into an `if` statement.

`decidr` skips that. It gives the model your options, runs **one forward pass**, and reads the probability of each option directly out of the model's own logits. No answer sentence. No JSON to repair. No decoding loop.

```python
from decidr import Client

client = Client(model="qwen3.5:4b")

decision = client.decide({
    "id": "route-1",
    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
    "question": "Which queue should handle this request?",
    "options": [
        {"id": "access",  "description": "Account access and authentication support."},
        {"id": "billing", "description": "Billing and payment support."},
        {"id": "sales",   "description": "Sales and product evaluation."},
    ],
})

decision.choice          # 'access'
decision.confidence      # 0.9999
decision.probabilities   # {'access': 0.9999, 'billing': 0.0001, 'sales': 0.0}
decision.is_reliable()   # True
```

Works with any model you already have in [Ollama](https://ollama.com). No fine-tuning, no extra runtime, no separate model to download.

## Install

```bash
pip install decidr
```

Requires Python 3.10+ and a running Ollama. Zero dependencies — it's stdlib `urllib` and `math`.

## How it works

Three things, in order:

**1. Options become letters.** Each option is presented to the model as `A`, `B`, `C`… rather than by its own name. This is not cosmetic: real labels like `billing` or `SYS_OUTAGE` are usually *several* tokens, and you cannot read a single-token probability for a multi-token string. Letters are single tokens in every vocabulary. The meaning lives in the descriptions, which is where the model actually reads it.

**2. One forward pass, no generation.** The prompt ends where the answer begins, and generation is capped at a single token. Reasoning mode is explicitly disabled — on a hybrid-reasoning model, a `<think>` preamble would put thinking tokens in the answer slot, and the next token would stop being the decision.

**3. The scores are read, not sampled.** Instead of taking whichever letter the model emitted, `decidr` reads the log probability of *every* option letter and normalizes over them. You get a distribution, not just a pick — so you can threshold on confidence, route ambiguous cases to a human, or log calibration over time.

## Two modes, picked automatically

How completely `decidr` can read those scores depends on your Ollama build. It probes once per process and tells you which mode you're in via `decision.mode`.

| Mode | When | Behavior |
|---|---|---|
| `ranked` | **Stock Ollama** (what you have today) | Falls back to `top_logprobs` (capped at 20 by the API). Options whose letters don't surface in that window are reported in `decision.unscored`. |
| `exact` | Ollama with [`logprob_tokens`](https://github.com/ollama/ollama/pull/18580) | Every option's probability is read directly from the full distribution, regardless of rank. |

**Why this distinction exists:** stock Ollama can only tell you about tokens that rank in the model's top 20 guesses. For a 3-option decision, the letters almost always make that cut and `ranked` is fine. For a 12-option decision, they often don't:

```
12 options, same model, same prompt:
  exact    ->  scored 12/12   reliable=True
  ranked   ->  scored 11/12   reliable=False   unscored: ['cat9']
```

`decidr` never invents a number for an option it couldn't measure. It reports it as unscored, `is_reliable()` returns `False`, and the remaining probabilities are normalized over what was actually observed. A fabricated floor value would be indistinguishable from a real measurement, which is the one thing a probability API must never do.

### Getting `exact` mode

`logprob_tokens` is a change I've proposed upstream to Ollama — **it is not merged yet**:

- Issue: [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579)
- PR: [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580)

Until it lands (if it lands), `decidr` works today in `ranked` mode against stock Ollama. Nothing here depends on that PR being accepted — `exact` mode is an upgrade, not a requirement. If you want it now, you can build from [the branch](https://github.com/devanmolsharma/ollama/tree/classification-logprobs).

## API

### `Client(model, host=..., timeout=..., temperature=..., force_mode=None)`

- `model` — any Ollama model name, e.g. `"qwen3.5:4b"`.
- `host` — defaults to `http://127.0.0.1:11434`.
- `temperature` — applied to the softmax over option logprobs, **not** to sampling. Higher values flatten confidence. Use this to calibrate against a labeled set (see below).
- `force_mode` — skip the capability probe and pin `"exact"` or `"ranked"`.

### `client.decide(row) -> Decision`

`row` needs `id`, `state`, `question`, and 2–16 `options`, each with an `id` and a `description`. `state` may be a string, dict, or list. Malformed rows raise `DecisionError` naming the problem rather than silently producing a confident wrong answer.

### `Decision`

| Field | |
|---|---|
| `choice` | option id with the highest probability |
| `confidence` | probability of `choice` |
| `probabilities` | option id → probability, sums to 1 **over scored options** |
| `logprobs` | raw log probabilities before normalizing |
| `mode` | `"exact"` or `"ranked"` |
| `unscored` | options the server could not report |
| `is_reliable()` | `False` when anything went unscored |
| `raw_answer` | the letter the model actually emitted |

`client.decide_all(rows)` runs a list sequentially.

## Calibration, honestly

The probabilities are **conditional on the option set you supplied** and are *not* calibrated out of the box. A raw instruction-tuned model's logits are not a calibrated probability distribution — they're systematically overconfident, and binary yes/no framings in particular carry a strong prior-driven skew.

What that means in practice: `choice` is usually trustworthy; `confidence` is directional, not a true probability. Before thresholding on it (e.g. "auto-route above 0.9"), fit `temperature` against a labeled sample from your own workload and measure whether the numbers track observed accuracy. Purpose-built decision models are trained against proper scoring rules precisely because this step matters.

If you need calibrated confidence out of the box more than you need "runs on anything you already have," use a trained decision model instead. `decidr`'s bet is that zero setup is worth more for most use cases, and that you should be told plainly where the limits are.

## What this is not

- **Not a new model.** It's a way of reading models you already run.
- **Not a fine-tune.** Nothing is trained; taxonomies change per request.
- **Not novel.** Reading option logits in a single pass is an established technique — [TypeSafe's Jev](https://www.typesafe.ai), [Laya](https://huggingface.co/convaiinnovations/laya), and [SemIf](https://github.com/TheoLeeCJ/SemIf) all do versions of it, and SemIf in particular has done far more rigorous benchmarking and calibration work than this has. `decidr`'s only claim is a narrow one: it's the smallest possible version that runs against an Ollama you already have, with zero dependencies and no model downloads.

## Limitations

- Sequential — no batching yet. One decision, one request.
- 2–16 options (letters `A`–`P`).
- `ranked` mode's completeness degrades as option count grows.
- Probabilities uncalibrated by default (see above).
- Tested against `qwen3.5:4b`. Small models (<1B) often won't treat a bare letter as a plausible next token; `decidr` raises a clear error rather than returning noise if none of the letters are scored.

## License

MIT
