Metadata-Version: 2.5
Name: decidr
Version: 0.3.0
Summary: Typed decisions from local LLMs in one forward pass, via Ollama
Project-URL: Homepage, https://github.com/devanmolsharma/decidr
Project-URL: Issues, https://github.com/devanmolsharma/decidr/issues
License: MIT
License-File: LICENSE
Keywords: classification,decisions,llm,logprobs,ollama,routing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# decidr

Typed decisions from local LLMs, in one forward pass. No generated sentence, no JSON to parse, no decoding loop.

```python
from decidr import Client

client = Client(model="qwen3.5:4b")

d = client.decide({
    "id": "route-1",
    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
    "question": "Which queue should handle this request?",
    "options": [
        {"id": "access",  "description": "Account access and authentication support."},
        {"id": "billing", "description": "Billing and payment support."},
        {"id": "sales",   "description": "Sales and product evaluation."},
    ],
})

d.choice          # 'access'
d.confidence      # 0.9779
d.probabilities   # {'access': 0.9779, 'billing': 0.0133, 'sales': 0.0088}
```

Works with any model already pulled in [Ollama](https://ollama.com). No fine-tuning, no extra runtime.

## Install

```bash
pip install decidr
```

Python 3.10+, a running Ollama. Zero dependencies.

## How it works

A model computes a probability over its whole vocabulary before it samples anything. Reading that distribution directly, instead of generating text and parsing it back into a decision, skips the entire decoding step. `decidr` sends the options, runs one forward pass, and reads the probability of each option straight out of the model's logits.

In `prefix` mode (the default), option ids are scored by their own text rather than a stand-in letter. Since there's no way to ask Ollama how a string tokenizes, boundaries are discovered by reading what the model actually returns and matching it against each option's id, one piece at a time; options that share a prefix get resolved together over more than one request. Full mechanism, including why every option is walked to the end of its own id rather than stopped as soon as it's distinguishable from the others: [docs/PREFIX_MATCHING.md](https://github.com/devanmolsharma/decidr/blob/main/docs/PREFIX_MATCHING.md).

## Two modes

| Mode | Default | Requests per decision | Option count |
|---|---|---|---|
| `prefix` | yes | 1, or more only when options collide | unlimited |
| `letters` | no | always 1 | up to 16 |

```python
Client(model="qwen3.5:4b")                    # prefix, the default
Client(model="qwen3.5:4b", id_mode="letters")  # fixed cost, capped option count
```

`letters` is the older, simpler path: each option is shown to the model as `A`, `B`, `C`, and the meaning stays in the description. It's a fixed one request per decision, always, but the option count is capped at 16 because there are only so many letters.

Both modes read scores from `top_logprobs`, capped at 20 by Ollama's API. If an option's real next token doesn't land in that top-20 window, it's reported in `decision.unscored` rather than assigned a guessed value. A separate change proposed upstream to Ollama removes that cap by letting a request name the exact tokens it wants scored, regardless of rank: [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579), [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580). Not merged yet. Both modes work today without it and pick it up automatically if it lands, since `decidr` probes for the capability once per process.

## Calibration

Probabilities are conditional on the option set supplied, and are not calibrated by default. `choice` is usually right. `confidence` on its own is not a true probability until it's checked against real outcomes.

`decidr.calibrate` fits one temperature to labeled data and rescales:

```python
from decidr.calibrate import fit_temperature, evaluate_out_of_fold

labeled = [(client.decide(row), row["correct_id"]) for row in labeled_rows]

fit_temperature(labeled)
# CalibrationResult(temperature=1.8, n=200, ece_before=0.15, ece_after=0.06, accuracy=0.83)

evaluate_out_of_fold(labeled, folds=5)
# fits T on 4 folds, scores it on the 5th, repeated for every fold.
# use this number, not fit_temperature's own -- a temperature can look
# better than it is by overfitting to the exact sample it was fit on.
```

Rescaling by a constant can't change which option wins, so `choice` stays fixed and only `confidence` moves. Without labeled data, `confidence` still ranks options correctly relative to each other; treat the absolute number as approximate.

## API

**`Client(model, host="http://127.0.0.1:11434", temperature=1.0, id_mode="prefix", force_mode=None)`**

`id_mode` is `"prefix"` or `"letters"`, see above. `temperature` scales the softmax over option logprobs, not sampling. `force_mode` skips the capability probe and pins `"exact"` or `"ranked"`.

**`client.decide(row) -> Decision`**

`row` needs `id`, `state`, `question`, and 2+ `options` (max 16 in `letters` mode), each with an `id` and a `description`. `state` may be a string, dict, or list. A malformed row raises `DecisionError` naming the problem.

**`Decision`**

| Field | |
|---|---|
| `choice` | option id with the highest probability |
| `confidence` | probability of `choice` |
| `probabilities` | option id → probability, sums to 1 over scored options |
| `logprobs` | raw log probabilities before normalizing |
| `mode` | `"prefix"`, `"exact"`, or `"ranked"` |
| `unscored` | options that could not be resolved |
| `is_reliable()` | `False` when anything went unscored |
| `raw_answer` | the model's own first reply |

`client.decide_all(rows)` runs a list sequentially.

## What's new

**v0.3** — real option ids by default (`prefix` mode), replacing the single-letter scheme as the default path. Removes the 16-option cap for the common case. Adds `decidr.calibrate` for fitting confidence against labeled data.

## Limitations

- Sequential, no batching.
- `prefix` mode's request count is variable: 1 for options that don't collide, more when they do, bounded by a depth cap.
- `letters` mode is capped at 16 options and always costs exactly 1 request.
- Both modes lose an option to `unscored` if its real next token falls outside `top_logprobs`'s top-20 window, which gets more likely as option count or ambiguity grows.
- Confidence uncalibrated by default; fixable with `decidr.calibrate` and labeled data.
- Tested against `qwen3.5:4b`. Small models (under 1B parameters) often won't treat a bare option id as a plausible next token at all.

## What this is not

- Not a new model. It's a way of reading models already running.
- Not a fine-tune. Nothing is trained; taxonomies change per request.
- Not novel as a technique. Reading option logits in a single pass is established; TypeSafe's Jev and [SemIf](https://github.com/TheoLeeCJ/SemIf) (an open-source alternative with far more benchmarking and calibration work than this project has) both do it. `decidr`'s claim is narrow: the smallest version that runs against an Ollama already installed, with zero dependencies.

## References

- [docs/PREFIX_MATCHING.md](https://github.com/devanmolsharma/decidr/blob/main/docs/PREFIX_MATCHING.md) — how prefix matching and chain-rule scoring work, in full
- [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579) — proposed upstream change removing the top-20 cap
- [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580) — the pull request implementing it
- [TheoLeeCJ/SemIf](https://github.com/TheoLeeCJ/SemIf) — open-source single-pass decision engine with calibration benchmarking
- [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) — a trained decision model doing a related version of this

## License

MIT
