Metadata-Version: 2.4
Name: inferrouter
Version: 0.1.0
Summary: InferRouter: loop-aware router and tracing for coding agents (Claude Code, Codex). Routes workflow steps and records cost/cache/verification/outcome receipts.
Author: Touchdown Labs
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: cryptography>=46
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"

# Touchdown InferGuard InferRoute MVP

**Status:** offline-demoable MVP with a live-capable GMI serverless replay path  
**Primary metric:** cost per accepted patch  
**Primary wedge:** Fable 5 / Claude Code / Codex / Cursor routing, cache, backend, and trace optimization

InferGuard plugs into a coding-agent workflow, records model/tool/context/cache/patch events, uses InferRoute for each workflow step, and outputs investor-demo artifacts:

```text
coding-agent workflow
→ trace
→ phase classifier
→ InferRoute static policy
→ InferRoute tiny-router shadow hook
→ cache-aware routing gate
→ backend allocator
→ GMI/GLM-5.2 serverless candidate replay
→ backend promotion plan
→ cost per accepted patch report
→ trace eligibility summary
→ post-training and endpoint handoff artifacts
```

This is not a generic proxy and not a model-hosting wrapper. It starts API/serverless-first and promotes traffic to rented/dedicated infrastructure only when traces prove utilization and quality.

## Quick demo (no API keys required)

```bash
cd inferguard
python -m td_inferguard.cli demo --out .td/demo
python -m checks.smoke_check
```

Outputs include:

```text
report.md                     14-section investor report
api_response.json             routes + analysis + eval gates
SKILL.md                      generated coding-agent skill
routing.policy.yaml           per-step route policy
backend_ladder.md             API → serverless → rented → owned → WorkloadPod
backend_promotion_plan.yaml   real gates computed from the trace (plan_only)
gmi_glm52_replay.json         replay plan (plan-only in the demo, always)
serverless_to_rented_break_even.md
tiny_router_shadow.md
oracle_ceiling.md
distillation_eligibility.md
tinker_job.yaml
prime_intellect_train_manifest.yaml
gmi_sidekick_endpoint_plan.md
terra_backend_plan.md
```

## CLI

```bash
python -m td_inferguard.cli init --target claude-code --provider gmi --out .td
python -m td_inferguard.cli skill write --target claude-code --out SKILL.md
python -m td_inferguard.cli demo --out .td/demo

python -m td_inferguard.cli route examples/sample_claude_code_run.jsonl --out .td/policies/routing.policy.yaml
python -m td_inferguard.cli report examples/sample_claude_code_run.jsonl --out .td/reports/inferguard.md

python -m td_inferguard.cli backend report --run examples/sample_claude_code_run.jsonl --out .td/reports/backend_ladder.md
python -m td_inferguard.cli backend promotion-plan --run examples/sample_claude_code_run.jsonl --out .td/backends/promotion_plan.yaml

python -m td_inferguard.cli replay gmi --run examples/sample_claude_code_run.jsonl --model zai-org/GLM-5.2-FP8 --out .td/replay/gmi_glm52_replay.json

python -m td_inferguard.cli live-check --out .td/live/live_check.json

python -m td_inferguard.cli serve --host 127.0.0.1 --port 8765
```

`td-inferguard` and `td-api` console scripts are available after `pip install -e .`; every command also works uninstalled via `python -m td_inferguard.cli`.

## API endpoints

```text
GET  /health
GET  /v1/version
GET  /usage/summary?by=dispatch_path      usage ledger summary
GET  /usage/events?limit=20               recent UsageEvent rows
GET  /usage/budget                        Fable 5 daily budget view
GET  /usage/limits                        Claude Code official or estimated limit view
POST /v1/route                      {"event": {...}} or a bare event object
POST /v1/optimize/claude-code        {"events": [...]} → analysis + routes + backend_summary + evals
POST /v1/backends/report            {"events": [...]} → tier costs/counts + promotion plan
POST /v1/backends/promotion-plan    {"events": [...]} → gate status from the trace
POST /v1/replay/gmi                 {"events": [...], "model": "zai-org/GLM-5.2-FP8"}
POST /v1/live-check                 {"frontier_model"?: "...", "sidekick_model"?: "..."} → real endpoint pings
GET  /v1/providers/check            dry configuration check, no requests
POST /v1/providers/check            {"live": true} → real per-provider pings
POST /v1/loop/audit                 {"events": [...]} → loop audit markdown
POST /v1/loop/blindspots            {"task": "..."} → blind-spot interview markdown
POST /v1/loop/merge-quiz            {"events": [...]} → merge quiz markdown
POST /v1/loop/ir                    {"events": [...]} → LoopIR snapshot
POST /v1/dataset/export             {"events": [...], "out"?: "dir"} → governed dataset export
POST /v1/workload/signature         {"events": [...]} → workload signature
POST /v1/kernels/opportunity-map    {"events": [...]} → kernel opportunity map (no speedup claims)
POST /v1/ingest/weka                {"dataset"|"source", "limit"} → Weka traces → TraceEvents
POST /v1/ingest/isb1                {"dataset"|"source", "limit"} → ISB-1 traces → TraceEvents
POST /v1/cache/simulate             {"events": [...]} → 64-token block cache report
POST /v1/benchmark/agentic-replay   {"events": [...], "profile": "..."} → replay report
POST /v1/benchmark/compare-real-vs-controlled  {"real_events": [...], "controlled_events": [...]}
POST /v1/reports/generate           {"events": [...], "out"?: "dir"} → 9 operator reports
```

## Usage and cost tracing

Cross-vendor coding-agent usage is recorded as append-only `UsageEvent` rows in
`inferguard/out/traces/usage-ledger.jsonl` or `TD_USAGE_LEDGER`. The full
runbook is `docs/inferguard/33-2026-07-07-cross-vendor-usage-cost-tracing.md`.
From the repo root, the two most useful commands are:

```bash
PYTHONPATH=inferguard python3 -m td_inferguard.cli usage run --dispatch-path codex --task-id TASK --trace-id TRACE -- codex exec --json "reply OK"
PYTHONPATH=inferguard python3 -m td_inferguard.cli usage import-claude
PYTHONPATH=inferguard python3 -m td_inferguard.cli usage report --by dispatch_path --json
```

Pricing follows the registry discipline: runtime-reported cost first, sourced
registry prices second, optional local overrides third, unknown otherwise.
Subscription rows are API-equivalent estimates, not billed dollars, and reports
keep them separate from actual API cost.

## Signal Plane (InferRoute semantic layer)

Every route builds a ContextPack (phase/risk/cache/history/privacy; prompt
hash + 200-char summary, never the raw prompt) and a Signal Plane readout
(heuristic/policy/cache/modality/semantic signals). Routing picks a role
first, then resolves a model alias from tdroute.yaml (`model_aliases:`).

```bash
td-inferguard eval routes --config tdroute.yaml --out out/reports/route_eval_report.md   # 9 deterministic policy cases, no live calls
td-inferguard mcp start --port 8790            # JSON-RPC tool server: inferguard.route_event, explain_route, build_context_pack, ...
td-inferguard route-receipt --run out/traces/proxy-events.jsonl --out out/route-receipt.json
```

Hook contracts (`td_inferguard/hooks.py`) give in-process agents pre/post
model-call, tool, test, patch, compaction, and human-decision integration.
All routing answers are `projected`; dollar figures in receipts are projected
from fixed cost bands until measured provider responses exist.

## Agentic benchmarking (real vs controlled)

```bash
td-inferguard ingest hf-weka --dataset semianalysisai/cc-traces-weka-062126 --split train --limit 10 --out out/traces/weka.jsonl
td-inferguard ingest isb1 --dataset wchen22/isb1-cc-traces --limit 10 --out out/traces/isb1.jsonl
td-inferguard cache simulate --run out/traces/weka.jsonl --out out/cache/weka.json
td-inferguard benchmark agentic-replay --run out/traces/weka.jsonl --profile with_subagents --out out/reports/agentic-replay.md --json out/reports/agentic-replay.json
td-inferguard benchmark compare-real-vs-controlled --real out/traces/weka.jsonl --controlled out/traces/isb1.jsonl --out out/reports/real-vs-controlled.md
td-inferguard receipts generate --run out/traces/weka.jsonl --out receipts/
```

Weka (SemiAnalysis) = real Claude Code production distribution; ISB-1
(`wchen22/isb1-cc-traces`) = controlled KV-cache stress. The source registry
(`td_inferguard/datasets/registry.py`, generated `source_registry.yaml`)
enforces lane separation: Weka never becomes a correctness eval, ISB-1 never
poses as real distribution, SWE-bench stays eval-only, execution-free
trajectories are never test-passing evidence. Mechanism quality gates
(`mechanisms.py`) refuse `supported` for lossy optimizations without a
completed quality eval. Replay-only traces prove runtime realism, not
accepted-patch quality.

## Evidence Spine

`td_inferguard/evidence/` is the append-only JSONL trace layer (validated event
types, claim_status, trace_eligibility, redaction_status). `td_inferguard/datasets/`
exports governed training data: routing/eval rows by default, SFT/RL rows only
when the trace carries explicit `sft_allowed`/`rl_allowed` rights; secrets are
redacted and every row carries its `source_bucket` so measured, synthetic, and
plan-only data never mix silently. `td_inferguard/workloads.py` turns traces
into workload signatures and a kernel opportunity map (measured bottlenecks vs
projected opportunities vs plan-only NVIDIA/AMD futures, never a speedup claim).

```bash
td-inferguard route-event --input event.json --out route.json
td-inferguard providers check [--live] [--out providers.json]
td-inferguard replay gmi [--live] --run <trace> --out <file>            # live is opt-in
td-inferguard replay openai-compatible [--provider openai|openrouter] [--live] --run <trace> --out <file>
td-inferguard dataset export --run <trace> --out out/dataset
td-inferguard workload signature --run <trace> --out sig.json
td-inferguard kernels opportunity-map --run <trace> --out kmap.json
td-inferguard loop ir --run <trace> --out loop_ir.json
td-inferguard reports --run <trace> --out out/reports
```

Provider env vars: `ANTHROPIC_API_KEY`/`ANTHROPIC_BASE_URL`/`TD_FRONTIER_MODEL`,
`GMI_API_KEY`/`GMI_BASE_URL`/`GMI_MODEL` (or `TD_SIDEKICK_MODEL`),
`OPENAI_API_KEY`/`OPENAI_BASE_URL`/`OPENAI_MODEL`,
`OPENROUTER_API_KEY`/`OPENROUTER_BASE_URL`/`OPENROUTER_MODEL`. Replay never
spends money without `--live`; `--live` without a key writes a `blocked`
artifact instead of pretending.

## Live endpoint check

```bash
export ANTHROPIC_API_KEY="sk-ant-..."   # frontier lane (Fable 5 via api.anthropic.com)
export GMI_API_KEY="..."                # sidekick lane (GLM-5.2 via api.gmi-serving.com)
python -m td_inferguard.cli live-check --out .td/live/live_check.json
```

The CLI also reads ignored local `.env` files from the current directory and the InferGuard package directory. Exported environment variables win over `.env`.

`live-check` sends one real, tiny request (`max_tokens=16`) per configured provider and writes a measured artifact with latency and token usage straight off the wire. Nothing is simulated: an unconfigured lane is recorded `not_configured` (no request made), a failed request is recorded `error` with the provider's actual response. Model defaults come from `TD_FRONTIER_MODEL` (default `claude-fable-5`) and `TD_SIDEKICK_MODEL` (default `zai-org/GLM-5.2-FP8`); nothing is hardcoded beyond those documented, overridable defaults.

## GMI serverless replay

```bash
export GMI_API_KEY="your key"                              # required for live replay
export GMI_BASE_URL="https://api.gmi-serving.com/v1"       # optional default
python -m td_inferguard.cli replay gmi --run <trace> --model zai-org/GLM-5.2-FP8 --out <file>
```

For local runs, `GMI_API_KEY`, `GMI_BASE_URL`, and `TD_SIDEKICK_MODEL` can also live in ignored `.env`.

Without the key the command writes a plan-only artifact and makes no requests. With the key it replays only sidekick-eligible phases (repo search, file/test-log summaries, docs formatting, low-risk first-fail repair) and labels results `candidate`, never `adopt`, until quality evals pass. High-risk patching and final review never replay by default. GLM-5.2 is a sidekick candidate; it does not replace Fable.

The provider layer (`td_inferguard/providers/`) is stdlib-only: an OpenAI-compatible client, a GMI subclass, and an Anthropic Messages API client for the frontier lane (minimal payload so the same request shape is valid on Fable 5, which rejects sampling and thinking params). Provider errors come back as structured objects, never exceptions.

## Tests

```bash
python -m pip install pytest    # or: pip install -e ".[dev]"
python -m pytest
```

Covers the CLI, demo outputs, phase classifier, InferRoute static policy, cache-switch gate, backend allocator, promotion gates, cost-per-accepted-patch analysis, the 14-section report, the eval gates, plan-only replay, and every API endpoint.

## Docs

- `docs/production-runbook.md`: offline demo, API server, GMI configuration, replay, real-trace import, report reading, and what not to claim yet.
- `../docs/inferguard/32-2026-07-06-inferroute-architecture-and-status.md`: canonical InferRoute architecture, policy gates, backend tiers, realtime sidecar, receipts, and honest status.

## Honesty labels

Artifacts carry `measured`, `estimated`, `projected`, `shadow_only`, `candidate`, `plan_only`, `adopt`, `reject`, or `inconclusive` labels. The demo does not claim live Fable savings, live Terra integration, live GMI deployment, a trained GLM-5.2/Qwen sidekick, or executed training jobs. InferRoute's tiny-router shadow is shadow-only; the static policy is what routes.
