Metadata-Version: 2.5
Name: thomas-train
Version: 0.2.4
Summary: Case→reward→train harness: baseline, fine-tune or RL, before/after — with gonogo as the decision layer.
Project-URL: Homepage, https://github.com/keppy/thomas
Project-URL: Source, https://github.com/keppy/thomas
Author: James Dominguez
License: MIT
License-File: LICENSE
Keywords: calibration,evaluation,fine-tuning,gonogo,llm,rl,training
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: gonogo-eval<0.5,>=0.4
Provides-Extra: dev
Requires-Dist: datasets>=3.0; extra == 'dev'
Requires-Dist: modal>=1.0; extra == 'dev'
Requires-Dist: numpy; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: scipy; extra == 'dev'
Requires-Dist: torch>=2.5; extra == 'dev'
Requires-Dist: transformers>=4.48; extra == 'dev'
Provides-Extra: encoder
Requires-Dist: datasets>=3.0; extra == 'encoder'
Requires-Dist: modal>=1.0; extra == 'encoder'
Requires-Dist: numpy; extra == 'encoder'
Requires-Dist: scipy; extra == 'encoder'
Requires-Dist: torch>=2.5; extra == 'encoder'
Requires-Dist: transformers>=4.48; extra == 'encoder'
Provides-Extra: post-train
Requires-Dist: tinker-cookbook<0.6,>=0.5; extra == 'post-train'
Requires-Dist: tinker<0.30,>=0.29; extra == 'post-train'
Provides-Extra: pretrain
Description-Content-Type: text/markdown

# thomas

Give thomas a case set and a reward function. It gives you a baseline card, a
training run, and a before/after comparison, with
[gonogo](https://github.com/keppy/gonogo) making the ship / don't-ship call at
each end. The domain supplies the case shape and the reward; thomas supplies
the plumbing.

*`thomas.train()`: named after Thomas the Tank Engine (it's a train) and Thomas
Barrow of Downton Abbey (it's the undercrogue that earns its place).*

```
                    ┌── encoder SFT ── (text, label) → Modal L4 → calibrated classifier
  Case set ──► thomas ┼── RL (Tinker) ── Case → TinkerEnv → LoRA RL → checkpoint
                    └── RL (Modal) ─── Case → TRL GRPO + vLLM colocate → LoRA adapter
                                  │
                                  └──► gonogo: verdict, interval, operating point
```

For RL tasks, the baseline, gradient reward and before/after eval use the
same domain `score_text`. Encoder classification is a separate exact-label
metric (`prediction == expected`); it does not call a Task's `score_text`.

## Where thomas sits

thomas is one tile of a small ecosystem that takes a task from *which model?*
to *ship it or not*:

```
your task ──► evalroute ─ choose a (model, effort) arm;
                │         measured where available, priors marked
                ▼
your cases ──► thomas ── train on labels (encoder) or score_text (RL);
                │         evaluate on untouched cases with gonogo
                ▼
              gonogo ── ship it, test a threshold on fresh cases, or walk away
```

- **[thomas](https://github.com/keppy/thomas)** (this repo) — the training
  harness. RL shares `score_text` across training and held-out evaluation;
  encoder SFT instead learns labels and needs a separate held-out evaluation.
- **[gonogo](https://github.com/keppy/gonogo)** — the decision layer, and the
  root of the map. Any agent, your real cases, a target; the verdict comes
  with the interval behind it.
- **[evalroute](https://github.com/keppy/hermes-plugin-evalroute)** — the
  routing layer: classify the task, hand back the arm with measured
  costs where available and priors otherwise; collect ratings to prioritize
  the next controlled batch.
- **The Hermes plugins** — the same three, inside your agent's session:
  [gonogo](https://github.com/keppy/hermes-plugin-gonogo) for decisions,
  [thomas](https://github.com/keppy/hermes-plugin-thomas) with a human approval
  hook on its training tool, and `/route` before the first turn.

[![Fine-tuning an encoder and getting a go/no-go verdict — thomas + gonogo](https://i.ytimg.com/vi/ozWITnaJtf4/maxresdefault.jpg)](https://youtu.be/ozWITnaJtf4)

**Video (40:33):** the whole pipeline worked live: plan, contract, a subagent building the thomas training path, two dead runs and one false alarm, and the Modal fine-tune of [ModernBERT-small-v2](https://huggingface.co/johnnyboycurtis/ModernBERT-small-v2) on an L4 for under $1, scored on the gonogo Banking77 canary. [Writeup](https://www.keppylab.com/blog/2026/09/21/banking77-canary-872-pass-two-dead-runs-one-false-alarm/).

- 87.2% [82.5%, 90.8%] pass rate against a 95% target → **AUTOMATE WITH REVIEW**
- at confidence ≥ 0.91: 98.3% precision [95.2%, 99.4%] on 71% of cases, the rest routed to a human
- calibration error 0.03 after temperature scaling; +10.0 points over the TF-IDF baseline on the same 250 cases (p < 0.001)

**Artifact availability:** the trained 146 MB Banking77 model and test-case
JSONL are local-only, ignored files; neither is in this repository or available
at an advertised download URL. No model has been published by this change.
The checked-in [250-case prediction receipt](examples/receipts/banking77_predictions.jsonl)
contains only case id, expected label, predicted label and calibrated confidence
(no customer text). `examples/banking77_receipt.py` replays the gonogo decision
from it; with your own original artifact and cases, it checks every CPU prediction
against the receipt. The [provenance manifest](examples/receipts/banking77_provenance.json)
identifies the locally checked model files, source CSVs, cases JSONL and receipt
by SHA-256; the script enforces those hashes when the local sources are present.
Hashes identify checked bytes, not independent attestation of a training run.
The receipt verifies the reported decision, **not** inference from a
public checkpoint. The dataset fetch/split logic is in
`examples/banking77_encoder.py`; Modal training costs money, and replay does not.
The operating threshold was selected on this same 250-case set and needs a
fresh holdout before deployment.

## Install

```bash
pip install thomas-train                 # core + gonogo
pip install "thomas-train[encoder]"      # + encoder fine-tune on Modal (torch, transformers, modal)
pip install "thomas-train[post_train]"   # + Tinker RL
```

Installs as `thomas-train` (the plain `thomas` name on PyPI belongs to an unrelated
project); imports as `thomas`. The examples live in the repo, so clone it to run them
(`pip install -e ".[encoder]"` from the clone).

## Try it

```bash
python examples/banking77_encoder.py --dry-run     # free: prints the config and the split, spends nothing
python examples/banking77_receipt.py             # replay the 250-case prediction receipt, CPU
# If you have the original local model and derived cases (not shipped):
python examples/banking77_receipt.py --model-dir examples/artifacts/banking77-enc \
    --cases-path examples/banking77_cases.jsonl   # verify all 250 predictions
python examples/banking77_predict.py --model-dir YOUR_LOCAL_ARTIFACT \
    "I lost my card"                              # CPU inference
```

## Encoder fine-tune (Modal)

The cheapest path, and the one behind the Banking77 result above: a small
encoder with a classification head, temperature-scaled on a held-out split so
its confidences mean something to gonogo.

```bash
# (text, label) JSONL in, artifact dir out
modal run -m thomas.encoder_train --model johnnyboycurtis/ModernBERT-small-v2 --data cases.jsonl

# the Banking77 canary end to end (dry run first, it's free)
python examples/banking77_encoder.py --dry-run
python examples/banking77_encoder.py
```

The artifact is a HuggingFace `save_pretrained` dir plus `label2id.json`,
`temperature.json` and `metrics.json` (see [CONTRACT.md §4](docs/CONTRACT.md#4-encoder-artifact)).
`examples/banking77_predict.py` runs it interactively on CPU.

## RL on a domain

```python
import thomas
from dataclasses import replace
from thomas.domains import tutor, tutor_scenarios

task = thomas.Task(
    name="tutor",
    cases=tutor_scenarios.CASES,
    score_text=tutor.score,           # (case, text) -> (reward, detail)
    oracle_reply=tutor.oracle,        # a reply that scores 1.0
    system_prompt=tutor.TUTOR_SYSTEM_PROMPT,
    render_messages=tutor.render,
    case_id=tutor.case_id,
)
assert task.oracle_check()            # the env is winnable by construction
# Illustrative only: for a real interval use substantially more untouched cases.
train_task = replace(task, cases=task.cases[:-2])
eval_task = replace(task, cases=task.cases[-2:])
# Costs Tinker credits; get operator approval before invoking.
run = thomas.post_train(train_task, eval_task=eval_task, max_steps=5,
                        score_text_fn="thomas.domains.tutor.score",
                        oracle_reply_fn="thomas.domains.tutor.oracle",
                        render_messages_fn="thomas.domains.tutor.render",
                        case_id_fn="thomas.domains.tutor.case_id",
                        from_json_fn="thomas.domains.tutor.cases_from_json")
print(thomas.compare(run.before, run.after).summary())
```

`post_train` and every Modal entry point spend money. Inspect the config and
approve **before invoking** these APIs; printing a config is not an approval
gate. `post_train` requires a disjoint `eval_task` and rejects overlapping
case ids or rendered prompts. It also requires `score_text_fn` to resolve to
the exact `Task.score_text` callable used for the baseline. The before/after
cards are measured on that untouched eval set, not training cases.

### Example domains

- `thomas.domains.tutor`: a tutoring trace scored by deterministic hard checks.
  The tutor must not assert unverified correctness or leak the answer, must
  keep the problem's conditions visible, and gets no reward for solving it for
  the student. Ten single-turn math scenarios in `tutor_scenarios`.
- `thomas.domains.skill_classify`: label a problem with a skill id, exact match.
- `thomas.domains.eerie`: literary continuation scored by n-gram overlap (the
  pretrain-path eval).

## Modal GRPO (custom checkpoints)

```bash
modal run -m thomas.trl_grpo --domain tutor \
    --model-path Qwen/Qwen3-4B --max-steps 5 \
    --wandb-name my-run
```

Saves a LoRA adapter to a Modal volume; metrics go to the
W&B project `thomas` (Modal secret `wandb`). The programmatic
`run_grpo_modal(gpu=...)` path is fixed to A100-40GB and rejects other GPU
names; it does not dynamically change Modal's allocation.

Known constraints (each cost a run to learn):

- **Colocate memory:** vLLM's budget is `total_gpu × vllm_gpu_memory_utilization`
  *minus* everything already resident in the process (the trainer model). The
  utilization must cover trainer weights + vLLM weights + KV cache. 0.75 on an
  A100-40GB works for 4B models; set `vllm_max_model_length` (TRL 1.13 kwarg —
  not `max_model_len`) to something small, or vLLM sizes KV for the model's
  native context.
- **Load in bf16** (`torch_dtype="auto"`): a silent fp32 load doubles the
  resident model and breaks the budget.
- **Reward functions receive chat format:** conversational completions arrive
  as a list of `{"role", "content"}` messages, not a string. `trl_grpo`
  extracts visible assistant text (strips `<think>`) on both local and Modal
  paths, then scores using the dataset's stable `case_id`. Matching the first
  case with the same prompt is wrong when distinct cases share wording.
- **VLM-class models (e.g. Qwen3.5) are broken upstream** in TRL colocate —
  weight-sync name remapping for `*ForConditionalGeneration` architectures
  (huggingface/trl#5269). thomas ships a monkeypatch prefix remap, but prefer
  text-only `ForCausalLM` checkpoints on this path.
- Server mode (`vllm_mode="server"`) hangs on Modal's kernel 4.19; colocate
  is the supported mode.

## The contract with gonogo

thomas and gonogo agree on six things: case ids, reward → `passed` / `score`,
the confidence definition (`max(softmax(logits / T))` over the full label set),
the artifact a training run leaves on disk, eval splits, and the gonogo calls
thomas depends on. They're written down and versioned in **[docs/CONTRACT.md](docs/CONTRACT.md)**. It's contract version 1,
tested against gonogo-eval `>=0.3,<0.4`.

```python
from thomas.contract import read_artifact   # stdlib only, no torch needed
art = read_artifact("examples/artifacts/banking77-enc")
art["temperature"], len(art["label2id"])    # 0.78, 77
```

## Works with

| | |
| --- | --- |
| [gonogo](https://github.com/keppy/gonogo) | Turns a score into a ship / don't-ship decision, with intervals |
| [thomas](https://github.com/keppy/thomas) | Trains the model: encoder fine-tune on Modal, or RL on Tinker / Modal |
| [hermes-plugin-gonogo](https://github.com/keppy/hermes-plugin-gonogo) | gonogo as Hermes agent tools |
| [hermes-plugin-thomas](https://github.com/keppy/hermes-plugin-thomas) | thomas's encoder path as Hermes agent tools, GPU launches behind the approval gate |

The thomas ↔ gonogo contract is [above](#the-contract-with-gonogo).

## What thomas is not

- Not gonogo (that's the scoring/decision layer; thomas imports it)
- Not a model (that's Tinker, Modal or nanogpt; thomas drives them)
- Not a case format (that's the domain)
- Not an eval framework (that's gonogo + the domain's checks)

## Changelog

[CHANGELOG.md](CHANGELOG.md).

## License

MIT
