# Provider dispatch

Verified against `unified_client.py`, `admission.py`, and provider adapters on
2026-08-20.

## Invariants

- **Callers dispatch the whole independent batch.** Features never pace paid AI
  work to protect a provider.
- **`UnifiedAIClient` owns admission.** Every catalog-routed chat, media, TTS,
  and extraction dispatch enters `admit_provider_call(profile)` immediately
  before touching the provider adapter.
- **Admission is credential-pool scoped.** The key is the resolved endpoint and
  base URL, not the feature, user, route, model display name, or browser tab.
- **Provider truth wins.** A classified HTTP 429 halves live concurrency and
  blocks the pool for `Retry-After`; successful calls add capacity back toward
  the ceiling.
- **Exact account limits are catalog data.** An offering may declare
  `metadata.provider_admission.{initial_concurrency,max_concurrency,requests_per_minute}`.
  Missing metadata uses adaptive admission (`32` initial, `512` ceiling), never
  a feature-local cap.
- **Queueing is unbounded at admission.** A batch of 100 creates 100 server
  tasks; the gate decides when each touches the provider.
- **Billing capture remains inside admission.** Waiting is free; once dispatch
  starts, `_dispatch_with_billing_net` still records or alarms on provider-billed
  failures.
- **First-use provider construction stays off the event loop.**
  `providers.__init__` exposes adapters through lazy `__getattr__`, and
  `UnifiedAIClient._get_provider_client()` resolves and constructs an uncached
  adapter through `asyncio.to_thread`. The process cache and construction lock
  preserve one client per provider under concurrent cold requests.
- **Lazy SDK clients are preflighted off-loop before every dispatch.**
  `_dispatch_with_billing_net()` calls `prepare_provider_clients()` before
  admission, so first construction and key-rotation rebuilds run in a worker
  thread. `keyed_provider_client` independently refuses cold construction on an
  active event loop.
- **Provider response decoding stays off the event loop.** Fastino streams raw
  response chunks asynchronously, then joins bytes, decompresses, and parses
  both the response envelope and nested entity JSON through
  `asyncio.to_thread`; a large compressed extraction response never freezes
  unrelated requests.
- **Per-call HTTP client construction stays off the event loop.** Fastino's
  fallback client path constructs TLS state through `asyncio.to_thread` before
  entering the async client context; injected shared clients retain connection
  pooling.
- **Retired wire aliases fail closed before dispatch.** Google native image generation
  repairs the obsolete `image_size="0.5K"` to the model-supported `512` or `1K`
  equivalent with a loud alarm; catalog truth still owns the normal translation.
- **Structured output wins cross-field conflicts.** Cerebras and Groq Chat
  endpoints support tools and structured output separately but reject them in
  one request. `UnifiedAIClient` keeps `json_object` or `json_schema`, loudly
  clears the effective tool surface, and preserves the authored declarations
  for a later compatible request.

## Entry points

- `admission.py` — adaptive endpoint pool, optional RPM window, snapshots.
- `unified_client.py::_dispatch_with_billing_net` — the mandatory dispatch seam.
- `errors.py::classify_provider_error` — normalized 429 and `Retry-After` facts.
- `orchestrator/overload_reroute.py` — retry/fallback after a provider refusal;
  admission prevents the retry storm before that policy runs.
- `orchestrator/parallel_executor.py` and `concurrent_engine.py` — batch
  coordination and per-item recovery only; their legacy concurrency knobs are
  compatibility fields and do not pace initial provider dispatch.

## Change log

- **2026-08-20** — Added the shared Cerebras/Groq tools-plus-structured-output
  conflict gate with forcing tests for both incompatible wire formats and the
  compatible/no-op cases.
- **2026-08-18** — Moved lazy provider SDK construction and key-rotation
  rebuilds off-loop and added a fail-closed cold-access guard.
- **2026-08-18** — Isolated Fastino fallback HTTP/TLS client construction from
  the request event loop and pinned the boundary with a responsiveness test.
- **2026-08-18** — Moved Fastino response decompression and JSON parsing off the
  event loop while preserving async network I/O and shared-client reuse.
- **2026-08-18** — Isolated cold provider SDK imports and constructors from the
  request event loop while preserving lazy, process-wide client reuse.
- **2026-08-18** — Added provider-boundary repair for Google's retired `0.5K`
  native-image size alias.
- **2026-08-18** — Removed orchestration-level initial and retry-sweep worker
  caps; both batch executors now create the full independent batch immediately.
- **2026-08-17** — Added universal adaptive provider admission and moved
  provider protection out of feature/browser batch limits.
