Answer choose and decide with a System One decision model, and fall back to the LLM
lingo asks its LLM to route between skills, pick tools and answer yes/no questions, and each of those calls generates a chain of thought before the answer. A System One decision model answers the same typed questions with a probability per option, in one forward pass and without generating text. This spec adds a SystemOne client that speaks the System One wire contract, and lets Engine.choose, Engine.decide and Engine.equip use it first and the LLM only when it is unsure.
01What this delivers
A contract client, not a vendor SDK
SystemOne(model, url, api_key) posts model, state and questions and reads answers. Any server that speaks the contract fits behind the URL: a hosted gateway or a local decision server. lingo imports nothing from either.
Cheaper routing with no code change
Lingo(llm=..., decider=SystemOne(...)) is the only change an app makes. Skill routing, Route, Decide, Choose and Act nodes go through the decider, because they already call the Engine methods it backs.
The LLM stays the safety net
An answer below the confidence threshold, or a decider that fails to answer, falls through to today's LLM path unchanged. The threshold is the knob between cost and how often the LLM is consulted.
The measurements come from a customer-service bot built on lingo: 40 messages over five routes (documents, store locations, products, off-topic, greetings), including six off-topic and manipulation attempts, all routed correctly.
02Decisions
maintainer marks a decision the maintainer made. proposed marks one this spec proposes.
| Question | Decision | Why |
|---|---|---|
| What lingo depends on | The wire contract only: an HTTP POST through httpx, already installed with openai | maintainer lingo knows nothing about any decision toolkit or vendor. Local models are reached through a server that speaks the contract. |
| Endpoint | A full url, defaulting to OpenRouter's https://openrouter.ai/api/alpha/decisions | proposed The same body works at OpenRouter's path and at /v1/systemone on other servers; a full URL covers both without a path option. |
| Which calls use it | choose, decide and equip | proposed Each is a typed question with a closed answer set. create and infer produce open values, which a decision model does not. |
| Threshold | One value per SystemOne, default 0.8, on the answer's confidence | proposed Confidence is the chosen option's probability for choice and max(p, 1 − p) for noul, so one number reads the same for both. |
| Failure | An HTTP or contract error falls back to the LLM and is logged | proposed The decider is an optimisation. An outage of the gateway must not take the app down with it. |
| State | The last history user and assistant messages (default 6), rendered as role: text lines | proposed System prompts describe the bot, not the case; decision models have a short window (the local ones accept about 1,000 tokens). |
| Option descriptions | A flow's or tool's description, else str(option) | proposed Skill docstrings already describe each route for the LLM; the decider gets the same text as its criteria. |
| Raw access | Engine.ask_many(context, questions) and SystemOne.decide(state, questions) return every answer with its probabilities | proposed An app that wants several questions in one pass (route, which searches, which place) asks them together and applies its own thresholds. |
03The API
from lingo import Lingo, LLM
from lingo.decider import SystemOne, choice, noul, score
decider = SystemOne(
model="typesafe/jev-1.14",
api_key=os.environ["OPENROUTER_API_KEY"], # or env SYSTEMONE_API_KEY
threshold=0.8,
)
bot = Lingo(llm=LLM(...), decider=decider) # routing now asks the decider first
# Raw, several questions in one request
decision = await decider.decide(
"¿Dónde hay tiendas en Holguín?",
{
"route": choice("¿De qué trata el mensaje?", {"stores": "ubicación de tiendas", "other": "otra cosa"}),
"urgent": noul("¿Necesita respuesta hoy?"),
"tone": score("¿Qué tan molesto está?", ["calmado", "molesto", "furioso"]),
},
)
decision["route"].choice # 'stores'
decision["route"].confidence # 0.99
decision["urgent"].noul # probability of yes
decision.usage.cost # as reported by the server, when it reports one
| Name | What it is |
|---|---|
| SystemOne | The client. model, url, api_key (env SYSTEMONE_API_KEY, then OPENROUTER_API_KEY), threshold, history, timeout, on_decision, and an optional httpx.AsyncClient for tests. |
| choice / noul / score | Builders for the three question types, returning plain dicts in the wire shape. |
| Decision | answers by question id, model as served, usage; indexable by id. |
| Answer | kind, choice, noul, score, probabilities, confidence. |
| DeciderError | Raised by SystemOne.decide on HTTP errors and answers that do not match the contract. |
04How the Engine uses it
Engine carries the decider
Engine(llm, tools, decider=None); scope() passes it on, and Lingo builds its Engine with self.decider and registers it for injection like the LLM.
choose and equip
Build criteria from the options (name to description), send one choice with the joined instructions, and return the mapped option when confidence ≥ threshold. Otherwise run the existing LLM path.
decide
One noul; true when p ≥ 0.5, accepted when max(p, 1 − p) ≥ threshold.
Fallback
A DeciderError or a low confidence is logged at debug level with the question and the confidence, and the LLM answers as it does today. on_decision receives every Decision, so an app can count how often each path answered.
Option names must be unique strings, as they already are for the LLM path, which keys its Literal on str(option).
05The wire contract
POST {url}
Authorization: Bearer {api_key}
{"model": "typesafe/jev-1.14",
"state": "user: ¿Dónde hay tiendas en Holguín?",
"questions": {
"q": {"type": "choice", "instructions": "...",
"criteria": {"stores": "ubicación de tiendas", "other": "otra cosa"}},
"u": {"type": "noul", "instructions": "..."},
"t": {"type": "score", "instructions": "...", "criteria": ["calmado", "molesto", "furioso"]}}}
200 {"model": "typesafe/jev-1.14-20261014",
"answers": {
"q": {"type": "choice", "choice": "stores", "probabilities": {"stores": 0.99, "other": 0.01}, "confidence": 0.99},
"u": {"type": "noul", "noul": 0.06},
"t": {"type": "score", "score": 0.4, "legend": {"0": "calmado", "1": "molesto", "2": "furioso"},
"probabilities": {"0": 0.62, "1": 0.36, "2": 0.02}, "confidence": 0.62}},
"usage": {"input_tokens": 826, "output_tokens": 0, "cost": 3.5e-05}}
This is the shape OpenRouter returned for typesafe/jev-1.14 on 2026-10-21. output_tokens was 61 on a three-question request and the reported cost matched input tokens alone.
A noul answer has no confidence field; the client derives it. The client tolerates a missing usage and unknown extra fields. box questions and image states are outside this change.
06Testing
- Unit tests with
httpx.MockTransport: request body shape for each question type; parsing of each answer type; derivednoulconfidence; HTTP 4xx/5xx and malformed answers raiseDeciderError. - Engine tests with a fake decider and
MockLLM: a confidentchoose,decideandequipmake no LLM call; a low-confidence one and a failing decider make exactly one; criteria carry flow and tool descriptions; the state holds only the lasthistoryuser and assistant messages. - A
Lingotest with two skills routes on the decider alone. - A live test against OpenRouter, skipped without
OPENROUTER_API_KEY, like the existing live tool-calling test.
07Non-goals and open questions
- Not in scope: extraction of open values (
create,infer),boxquestions, images, batching several states in one request. - Not in scope: a public
Engine.chatwith tools, which apps reach today throughengine._llm; it is a separate change. - Open: whether
Routeshould expose the decision so a skill can read the confidence it was chosen with. - Open: whether OpenRouter's path, still under
/api/alpha/, moves when the endpoint leaves alpha. The default URL is one constant.