Metadata-Version: 2.5
Name: decidr
Version: 0.4.0
Summary: Typed decisions from local LLMs in one forward pass, via Ollama
Project-URL: Homepage, https://github.com/devanmolsharma/decidr
Project-URL: Issues, https://github.com/devanmolsharma/decidr/issues
License: MIT
License-File: LICENSE
Keywords: classification,decisions,llm,logprobs,ollama,routing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# decidr

Typed decisions from local LLMs, in one forward pass. No generated sentence, no JSON to parse, no decoding loop.

```python
from decidr import Client

client = Client(model="qwen3.5:4b")

d = client.decide({
    "id": "route-1",
    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
    "question": "Which queue should handle this request?",
    "options": [
        {"id": "access",  "description": "Account access and authentication support."},
        {"id": "billing", "description": "Billing and payment support."},
        {"id": "sales",   "description": "Sales and product evaluation."},
    ],
})

d.choice          # 'access'
d.confidence      # 0.9779
d.probabilities   # {'access': 0.9779, 'billing': 0.0133, 'sales': 0.0088}
```

Works with any model already pulled in [Ollama](https://ollama.com). No fine-tuning, no extra runtime.

## Install

```bash
pip install decidr
```

Python 3.10+, a running Ollama. Zero dependencies.

## How it works

A model computes a probability over its whole vocabulary before it samples anything. Reading that distribution directly, instead of generating text and parsing it back into a decision, skips the entire decoding step. `decidr` sends the options, runs one forward pass, and reads the probability of each option straight out of the model's logits.

In `prefix` mode (the default), option ids are scored by their own text rather than a stand-in letter. Since there's no way to ask Ollama how a string tokenizes, boundaries are discovered by reading what the model actually returns and matching it against each option's id, one piece at a time; options that share a prefix get resolved together over more than one request. Full mechanism, including why every option is walked to the end of its own id rather than stopped as soon as it's distinguishable from the others: [docs/PREFIX_MATCHING.md](https://github.com/devanmolsharma/decidr/blob/main/docs/PREFIX_MATCHING.md).

## Two modes

| Mode | Default | Requests per decision | Option count |
|---|---|---|---|
| `prefix` | yes | 1, or more only when options collide | unlimited |
| `letters` | no | always 1 | up to 16 |

```python
Client(model="qwen3.5:4b")                    # prefix, the default
Client(model="qwen3.5:4b", id_mode="letters")  # fixed cost, capped option count
```

`letters` is the older, simpler path: each option is shown to the model as `A`, `B`, `C`, and the meaning stays in the description. It's a fixed one request per decision, always, but the option count is capped at 16 because there are only so many letters.

Both modes read scores from `top_logprobs`, capped at 20 on stock Ollama. If an option's real next token doesn't land in that window, it's reported in `decision.unscored` rather than assigned a guessed value. That cap is Ollama's own request validation, not a limit either backend actually has — checked directly, not assumed, by building a server with it raised and confirming a full 100-alternative response comes back correctly. A change proposed upstream raises it from 20 to 100: [ollama/ollama#18590](https://github.com/ollama/ollama/issues/18590), [ollama/ollama#18591](https://github.com/ollama/ollama/pull/18591). Not merged yet.

`Client` asks for 100 alternatives by default. Against stock Ollama that request is rejected once, and the client falls back to 20 and remembers it for the rest of that `Client`'s life — one failed request the first time a decision is made, never repeated. Against a server with the cap raised, it works on the first try. Either way, nothing needs to be configured by hand.

## Calibration

Probabilities are conditional on the option set supplied, and are not calibrated by default. `choice` is usually right. `confidence` on its own is not a true probability until it's checked against real outcomes.

`decidr.calibrate` fits one temperature to labeled data and rescales:

```python
from decidr.calibrate import fit_temperature, evaluate_out_of_fold

labeled = [(client.decide(row), row["correct_id"]) for row in labeled_rows]

fit_temperature(labeled)
# CalibrationResult(temperature=1.8, n=200, ece_before=0.15, ece_after=0.06, accuracy=0.83)

evaluate_out_of_fold(labeled, folds=5)
# fits T on 4 folds, scores it on the 5th, repeated for every fold.
# use this number, not fit_temperature's own -- a temperature can look
# better than it is by overfitting to the exact sample it was fit on.
```

Rescaling by a constant can't change which option wins, so `choice` stays fixed and only `confidence` moves. Without labeled data, `confidence` still ranks options correctly relative to each other; treat the absolute number as approximate.

## API

**`Client(model, host="http://127.0.0.1:11434", temperature=1.0, id_mode="prefix")`**

`id_mode` is `"prefix"` or `"letters"`, see above. `temperature` scales the softmax over option logprobs, not sampling.

**`client.decide(row) -> Decision`**

`row` needs `id`, `state`, `question`, and 2+ `options` (max 16 in `letters` mode), each with an `id` and a `description`. `state` may be a string, dict, or list. A malformed row raises `DecisionError` naming the problem.

**`Decision`**

| Field | |
|---|---|
| `choice` | option id with the highest probability |
| `confidence` | probability of `choice` |
| `probabilities` | option id → probability, sums to 1 over scored options |
| `logprobs` | raw log probabilities before normalizing |
| `mode` | `"prefix"` or `"ranked"` |
| `unscored` | options that could not be resolved |
| `is_reliable()` | `False` when anything went unscored |
| `raw_answer` | the model's own first reply |

`client.decide_all(rows)` runs a list sequentially.

## What's new

**v0.4** — removed the `exact`/`logprob_tokens` code path. It depended on a change proposed upstream to Ollama that was later withdrawn in favor of a simpler one ([ollama/ollama#18590](https://github.com/ollama/ollama/issues/18590)/[#18591](https://github.com/ollama/ollama/pull/18591), replacing the closed #18579/#18580), so no server would ever produce it. `mode` is now always `"prefix"` or `"ranked"`.

**v0.3** — real option ids by default (`prefix` mode), replacing the single-letter scheme as the default path. Removes the 16-option cap for the common case. Adds `decidr.calibrate` for fitting confidence against labeled data.

## Limitations

- Sequential, no batching.
- `prefix` mode's request count is variable: 1 for options that don't collide, more when they do, bounded by a depth cap.
- `letters` mode is capped at 16 options and always costs exactly 1 request.
- Both modes lose an option to `unscored` if its real next token falls outside `top_logprobs`'s top-20 window, which gets more likely as option count or ambiguity grows.
- Confidence uncalibrated by default; fixable with `decidr.calibrate` and labeled data.
- Tested against `qwen3.5:4b`. Small models (under 1B parameters) often won't treat a bare option id as a plausible next token at all.

## What this is not

- Not a new model. It's a way of reading models already running.
- Not a fine-tune. Nothing is trained; taxonomies change per request.
- Not novel as a technique. Reading option logits in a single pass is established; TypeSafe's Jev and [SemIf](https://github.com/TheoLeeCJ/SemIf) (an open-source alternative with far more benchmarking and calibration work than this project has) both do it. `decidr`'s claim is narrow: the smallest version that runs against an Ollama already installed, with zero dependencies.

## References

- [docs/PREFIX_MATCHING.md](https://github.com/devanmolsharma/decidr/blob/main/docs/PREFIX_MATCHING.md) — how prefix matching and chain-rule scoring work, in full
- [ollama/ollama#18590](https://github.com/ollama/ollama/issues/18590) — proposed upstream change raising the top_logprobs cap from 20 to 100
- [ollama/ollama#18591](https://github.com/ollama/ollama/pull/18591) — the pull request implementing it
- [TheoLeeCJ/SemIf](https://github.com/TheoLeeCJ/SemIf) — open-source single-pass decision engine with calibration benchmarking
- [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) — a trained decision model doing a related version of this

## License

MIT
