Calibration instrument pre-alpha

Decisions that come with
honest numbers

Exu trains encoder-only models that answer a typed question with a probability distribution, not a paragraph. The reward is a strictly proper scoring rule, so the only way to raise it is to be honest about the odds.

Try the training loop Source and docs

01 The failure

A reward of 1 or 0 teaches a model to lie with confidence

Reward the model only for being right and its best move is to put all the mass on the most likely option. Accuracy goes up, calibration falls apart. A scoring rule fixes this: its expected value is highest when the reported odds are the true odds.

Two options, true odds 70 / 30 drag the confidence on the true option
honest optimum
log score
0.000
spherical
0.000
binary reward
0.000

02 The contract

One mechanism, three hats

Options are written at request time and read as text, so a new task is a new question, not a new output layer. noul is a choice with two fixed options; score is a choice over ordinal levels.

state

question

answer

03 The sequence

One sequence, one marker per option

The model reads the options as text. A mask marker in front of each option is where its score is read from, and because the encoder is bidirectional, that position already sees the option, its rivals, the instruction and the state.

  • instruction
  • option text
  • markers and separators
  • state (remainder)

04 The loop

Sample the logits, reward the candidates, step

The policy perturbs its own logits instead of generating tokens. Perturbations that scored above the group baseline pull the logits toward themselves. This is the real update rule, running here on three options.

Target: option B sigma anneals as you train
logits
8 sampled candidates
step 0
mean reward
n/a
advantage spread
n/a
P(option B)
0.333

05 Do it

From a labeled file to a portable checkpoint

Train the direct baseline first, then the policy version, and keep the policy version only if it wins on held-out ECE or NLL. Calibrate on held-out data and ship the temperature map inside the checkpoint.

install and train 
uv sync --extra dev

exu-train \
  --mode rlcd \
  --train data.jsonl --train-split train \
  --validation data.jsonl --validation-split validation \
  --calibration data.jsonl --calibration-split calibration \
  --output artifacts/my-model \
  --encoder google-bert/bert-base-multilingual-cased \
  --epochs 4 --batch-size 8 \
  --option-shuffle --calibrate
evaluate and serve 
exu-evaluate \
  --checkpoint artifacts/my-model \
  --data data.jsonl --split test \
  --order-permutations 4 --latency

# then, in Python
from exu import DecisionRuntime
runtime = DecisionRuntime.load("artifacts/my-model")
decision = runtime.decide(state, question)
print(decision.label, decision.confidence)
exit criteria, in order
  1. Sequence builder tested: truncation, many options, marker out of range, injected mask token.
  2. Encoder chosen after measuring tokenizer fertility on your own text.
  3. Held-out split by whole task family, never by example.
  4. Direct baseline beats the per-question prior with margin.
  5. The RLCD policy beats the direct baseline on held-out ECE or NLL, otherwise it does not ship.
  6. Temperatures fitted on held-out data, no value sitting on a bound.
  7. Order robustness and selective coverage reported beside accuracy.