Engineering audit  ·  No. 2026-07-20

Is the per-task cost meter honest?
An audit of KISS Sorcar's budget accounting

§1What was audited

Every tool result in a KISS Sorcar task ends with a line like Budget: $12.48/$1000.00. That number is computed by a four-stage pipeline, and this audit walked all four stages end to end:

Provider response usage: tokens by kind Per-provider extractor (input, output, cache-read, cache-write, 1h-write) calculate_cost() model_info.py budget_used hard cap enforced prices from MODEL_INFO.json (532 models) + cache multipliers + long-context tiers
Fig. 1 — The budget pipeline. Each model wrapper normalizes provider usage into five token counts; calculate_cost prices them per million tokens.

Prices were checked against the primary pricing pages of OpenAI (developers.openai.com, incl. four per-model pages), Anthropic (platform.claude.com), Google (ai.google.dev), DeepSeek (api-docs.deepseek.com), Moonshot (platform.kimi.ai — overview, K3, K2.7-Code, K2.6, K2.5), xAI (docs.x.ai), and OpenRouter (kimi-k3 and kimi-k2.5 listings) — ten pages across seven provider sites.

§2What checks out

The token accounting itself is sound. No double-billing paths were found:

Model (KISS entry)KISS in/out $/MPublishedVerdict
claude-fable-510 / 50$10 / $50 · cache $1 / $12.50 / $20correct
claude-opus-4-8 · -4-7 · -4-6 · -4-55 / 25$5 / $25correct
claude-sonnet-52 / 10$2 / $10 intro (→ $3/$15 on Sep 1, 2026)correct today
gpt-5.6-sol / -terra / -luna5/30 · 2.5/15 · 1/6same, + cache-write 1.25× + long tiercorrect
gpt-5.5 / gpt-5.4 (+mini, nano)5/30 · 2.5/15 …same, free cache writescorrect
gemini-3.1-pro-preview2 / 12$2/$12, >200k $4/$18, cache 0.1×correct
gemini-3.5-flash · 2.5-pro · flash tiers1.5/9 · 1.25/10 …samecorrect
deepseek-v4-pro / v4-flash (OpenRouter)0.435/0.87 · 0.098/0.196hit ratios 0.003625/0.435 and 0.02correct
grok-4.5 (openrouter/x-ai)2 / 6$2 / $6correct

§3Five real problems, found and fixed

BUG 1 · STALE PRICES

Direct Moonshot (Kimi) prices were out of date — undercharging up to 37%

platform.kimi.ai's July-2026 price sheets no longer match what shipped in MODEL_INFO.json, and Kimi K3 (launched Jul 16) was missing entirely, which made any kimi-k3 run die with a KISSError: unknown model.

ModelWas (in/out $/M)Actual (hit / miss / out)Now
kimi-k2.50.60 / 2.500.10 / 0.60 / 3.000.60 / 3.00, hit 0.10
kimi-k2.60.60 / 2.500.16 / 0.95 / 4.000.95 / 4.00, hit 0.16
kimi-k2.7-code0.60 / 2.500.19 / 0.95 / 4.000.95 / 4.00, hit 0.19
kimi-k3missing0.30 / 3.00 / 15.00added, 1M ctx
Output price, $ per 1M tokens — registry before vs. provider's actual 051015 kimi-k2.5 kimi-k2.6 kimi-k2.7-code (absent) kimi-k3 registry, before provider, actual
Fig. 2 — Registry output prices trailed Moonshot's July-2026 price sheet; K3 wasn't priced at all.
BUG 2 · WRONG THRESHOLD

OpenAI long-context tier fired at 200k — the real line is 272k input tokens

OpenAI's model pages state, verbatim: “Prompts with >272K input tokens are priced at 2x input and 1.5x output for the full request.” The code switched gpt-5.4 / 5.5 / 5.6 to double rates at 200,000 tokens — and it counted output tokens toward the threshold too. Both errors overcharge. Example: a gpt-5.6-sol call with a 250k-token prompt and 100k output was billed $7.00 instead of $4.25 (+65%).

Fix: the tier now keys off prompt-side tokens only (fresh input + cache reads + cache writes), with the OpenAI threshold at 272,000 and Gemini's at its documented 200,000 (“$2.00, prompts ≤ 200k tokens”). Boundary covered by tests at 272,000 vs 272,001.

When does a gpt-5.6 request cross into 2× pricing? 0 100k 200k 300k 400k old code: 200k, input+output fixed: 272k, input only requests here were billed 2× in, 1.5× out — wrongly Gemini Pro models keep their documented 200k prompt threshold — that one was right all along.
Fig. 3 — The 200,000→272,000 correction, and the band of prompts that used to be overbilled.
BUG 3 · OVERCHARGE 2.5×

OpenRouter kimi-k3 cache reads priced by a generic 0.25× rule

openrouter/moonshotai/* models fall under a blanket “cache read = 25% of input” rule. For kimi-k3, OpenRouter lists cache reads at $0.30/M (10% of the $3 input price); the rule was charging $0.75/M. Fixed with an explicit per-entry price — explicit JSON values always beat prefix heuristics.

CAUGHT IN REVIEW

Bug 4 — the first version of this very fix introduced a regression

The adversarial reviewer (gpt-5.6-sol) probed the patched registry and found that adding explicit cache-read prices made the pricing-defaults pass skip those entries entirely, leaving their cache-write price None — which calculate_cost conservatively bills at the full input rate. A million cache-write tokens on kimi-k2.5 suddenly cost $0.60 instead of the correct $0.00 (Moonshot has no cache-write fee). Fixed by writing explicit cache_write_price_per_1M: 0.0 into all five entries, with a test asserting the billed cost is exactly zero.

BUG 5 · DOCS DRIFT

A stale comment claimed Kimi cache hits cost 0.25× input

The code comment cited “K2.5: $0.15 hit vs $0.60 miss.” Current sheets show per-family ratios of 0.10×–0.20× (K2.5 $0.10, K2.6 $0.16, K2.7-Code $0.19, K3 $0.30). Comment rewritten; the 0.25× multiplier survives only as a conservative fallback for entries with no explicit value (e.g. legacy moonshot-v1).

§4The adversarial review

Per the task brief, all code changes were made by claude-fable-5, and the resulting diff plus the draft of this report were handed to gpt-5.6-sol for a strictly read-only cross-examination (it modified no files; its spend was ~$2 of a $200 review allowance — about 1%). Its findings, all incorporated:

§5Known limits, deliberately left alone

§6Verification

Files changed: src/kiss/core/models/MODEL_INFO.json (Kimi prices, kimi-k3, explicit cache read/write), src/kiss/core/models/model_info.py (272k OpenAI threshold, prompt-side tier decision, comment fix), src/kiss/tests/core/models/test_cost_accuracy_2026.py (updated + 8 new regression tests).

Evidence: the cost-accuracy suite and every other pricing test pass (219 tests across the five cost-touching files; 920+ in the full models suite), and uv run check (lint + mypy over 927 files) is green.

KISS Sorcar cost audit · July 20, 2026 · Sources: developers.openai.com, platform.claude.com, ai.google.dev, api-docs.deepseek.com, platform.kimi.ai, docs.x.ai, openrouter.ai. Development model: claude-fable-5 · adversarial reviewer: gpt-5.6-sol.