# Cost reconciliation and optimization — plan

**Status:** planning + **in-process capture implemented** (see `provider_cost_usd` on `llm_call_log`, dashboard **Cost analytics**). Optional webhook/Broadcast-to-custom-host remains **out of scope** for the shipped product unless needed later.

**Goal:** Replace guesswork with **ground-truth spend** from OpenRouter (from each API response), **compare** to internal table estimates, and drive a **prioritized optimization** loop — **without** requiring a separate webhook receiver for operators.

---

## 1. Why a single public “$/day” number is misleading

The static site no longer publishes a headline dollar-per-day total — real spend varies too much. When rough numbers appeared in older copies, they were **component guesses**. Actual bills are often higher because:

| Gap | Effect on bills |
|-----|-----------------|
| **`llm_call_log.estimated_cost_usd`** is derived from **`llm.py` `_COST_TABLE`** (token math), not OpenRouter’s charged amount | Table drift vs live OpenRouter pricing; provider routing; cached-token discounts not modeled |
| **OpenRouter `:online` / web search** | Extra charges beyond token estimates (see `docs/MODELS.md`) |
| **Tool loops** (`call_llm_chat_with_tool_loop`, Telegram callable skills) | Multiple completions per “user turn”; easy to under-count in mental models |
| **Reasoning / extended thinking** (`thinking_budget_tokens`, per-source routing) | Output tokens and provider rules differ from a simple in/out table |
| **`:exacto` and plugins** (response healing, context compression when enabled) | Additional provider/plugin behavior vs naive token × price |
| **Budget caps** use `SUM(estimated_cost_usd)` on `llm_call_log` | Protects against runaway *estimates*; does not guarantee match to the OpenRouter invoice |

**Today’s pipeline (simplified):** OpenRouter JSON → `usage.prompt_tokens` / `completion_tokens` → `_estimate_cost()` → SQLite `llm_call_log`. Any field in `usage` that reflects **actual billed cost** is logged in **text logs** (`usage=…` in `llm_timing` lines) but not stored as a first-class column for analytics.

---

## 2. Objectives

1. **Ground truth:** Persist or import OpenRouter **actual** cost per generation (from API response and/or account-level usage APIs).
2. **Reconcile:** Join provider-side events to `llm_call_log` rows using **`session_id`**, **`trace_id`**, and OpenRouter **Broadcast / OTLP** identifiers (see §4).
3. **Report:** Daily/weekly **variance** (estimate vs actual), **by `source`**, **by model**, and **by feature flags** (web, tools, reasoning).
4. **Optimize:** A short, actionable list: routing changes, caps, toggles, and code paths already documented in `docs/MODELS.md` and `docs/SETTINGS-CONSTANTS-AUDIT.md`.

---

## 3. Data sources (two lanes)

### Lane A — In-process (no extra host)

- **OpenRouter chat completion `usage` object:** OpenRouter documents usage accounting (tokens, and often **cost** / credits). **Plan:** extend `_record_llm_usage` / `llm_call_log` schema to store **provider-reported cost** when present, alongside today’s estimate (non-breaking: new nullable columns or a parallel table keyed by `llm_call_log` row id).
- **OpenRouter account / usage API (if applicable):** Periodic pull (cron or manual CLI) for **invoice-level** totals as a sanity check against summed row costs.

### Lane B — OpenRouter Broadcast → **webhook** (OTLP)

OpenRouter can POST **OTLP JSON** traces to a URL ([Broadcast → Webhook](https://openrouter.ai/docs/guides/features/broadcast/webhook)): spans include **model**, **token usage**, **cost**, timing, and **GenAI** attributes.

Furoshiki already sends **`session_id`** and **`trace`** (e.g. `trace_id`, generation name = `source=`) to OpenRouter for observability (`docs/MODELS.md`, `scripts/llm.py`). Those fields map into OTLP as documented (`session.id`, `trace.metadata.*`), which is ideal for **joining** webhook spans to `llm_call_log`.

**Recommended usage:**

- **Development / staging:** A small **public HTTPS** receiver (see §5) registered only in a **non-production** OpenRouter project or a **dev API key**, so production traffic is not duplicated unless intended.
- **Production (optional later):** Same pattern behind auth + rate limits; store raw payloads with retention and privacy controls.

---

## 4. Correlation model

**Join keys (in priority order):**

1. **`trace_id`** — already on `llm_call_log`; should match OTLP `traceId` / metadata when Broadcast is enabled.
2. **`session_id`** — scopes Telegram turns (`telegram:<chat_id>:<thread>`); narrows search when one trace has multiple spans.
3. **Time window + model + source** — fallback when ids differ (retries, partial failures).
4. **OpenRouter completion id** — if we add persistence from `llm_timing` logs (`completion_id=`), join to span attributes if exposed.

**Edge cases to design for:**

- **Multiple LLM calls per logical “turn”** (tool loops): one trace, many spans / rows — reconcile **sum of child costs** vs one `llm_call_log` row per HTTP completion (current behavior).
- **Dropped async rows** — `llm_call_log` queue full: webhook may still show spend; reconciliation should flag **orphan provider events**.
- **Privacy** — OpenRouter Privacy Mode strips prompt/completion content but keeps usage/cost; align with operator expectations.

---

## 5. Development-only webhook receiver (proposed shape)

Not a full spec — enough to align implementation later.

| Concern | Approach |
|--------|----------|
| **Exposure** | Separate process or `repair-api` optional route behind **`FUROSHIKI_*`** flag default **off**; or standalone FastAPI on `127.0.0.1` + **ngrok**/Cloudflare Tunnel for OpenRouter’s test |
| **Auth** | Custom headers from OpenRouter config + constant-time compare of bearer token; reject otherwise |
| **Payload** | `POST application/json` OTLP; validate structure; **idempotent insert** (dedupe by trace+span+timestamp hash) |
| **Storage** | New SQLite table under `FUROSHIKI_DATA` (e.g. `openrouter_otlp_spans`) or JSONL under `memory/` for dev; retention TTL |
| **Cross-reference** | Batch job: `SELECT` spans vs `llm_call_log` for a date range → **variance view** |

**Operator story:** “Point OpenRouter Observability → Webhook at `https://<tunnel>/openrouter/webhook` with header `Authorization: Bearer <secret>`; run `furoshiki cost reconcile --days 7` (future CLI).”

---

## 6. Analytics outputs

1. **Variance table:** `sum(estimated_cost_usd)` vs `sum(actual_cost_usd)` (or provider) by day and by `source`.
2. **Top drivers:** Sources and models with largest **absolute** and **percentage** variance.
3. **Feature flags:** Compare spend with `openrouter_web`, tool loops, reasoning-heavy sources.
4. **Trace flow:** Optional: export a **Sankey-style** or tabular “cost flow” from `source` → child spans (for debugging expensive paths).

---

## 7. Optimization strategy (after measurement)

Measure first; then apply levers already in the codebase:

- **Routing / models:** `llm-routing.json`, tier shortcuts, disable or narrow `:online` for Telegram or reflections.
- **Cadence:** Brain schedules, soul tick, outreach pulse — fewer calls beats smaller models when volume dominates.
- **Tool loop:** Reduce unnecessary rounds (prompt/tool schema size, early exit).
- **Context:** Shorter transcripts, context compression env (when acceptable for quality).
- **Budgets:** Tighten `memory/budget_settings.json` with alerts at lower `%` once estimates are calibrated to actuals.

Re-run reconciliation monthly after changes.

---

## 8. Phased delivery (suggested)

| Phase | Outcome |
|-------|---------|
| **P0** | Document actual vs estimate gap; add **provider `usage.cost`** (or equivalent) to DB when present; backfill script from existing logs if usage JSON was logged |
| **P1** | OTLP webhook receiver (dev) + storage + one **variance report** (CLI or dashboard tab) |
| **P2** | Join polish: completion id, orphan detection, dashboard charts |
| **P3** | Automated alerts (e.g. weekly Slack/Telegram if variance > N%) |

---

## 9. Risks and compliance

- **Secrets:** Webhook URL + tokens must not land in git; use env and operator docs.
- **PII:** Prefer Privacy Mode on Broadcast for shared endpoints; restrict raw payload access.
- **Double counting:** If both **inline API cost** and **webhook** persist, reconcile rules must avoid summing twice (single source of truth per generation).

---

## 10. References (in-repo)

- `scripts/llm.py` — `_estimate_cost`, `_record_llm_usage`, `llm_call_log`, OpenRouter Broadcast fields  
- `docs/MODELS.md` — web search, tiers, observability  
- `https://furoshiki.ai/` — cost section (update when reconciliation numbers exist)  
- OpenRouter: [Broadcast Webhook](https://openrouter.ai/docs/guides/features/broadcast/webhook), Usage / pricing docs

---

*Next step when implementing: update `docs/SELF-AWARENESS-DESIGN.md` and `docs/MODELS.md` in the same change set as schema/code, per repo documentation rules.*
