Metadata-Version: 2.4
Name: llm-rates
Version: 0.4.2
Summary: One vendored litellm-catalog snapshot + one public overlay + shared lookup/cost math for portfolio LLM pricing
Author-email: Mike Donnelly <82827803+m0j0d@users.noreply.github.com>
License: MIT
Project-URL: Homepage, https://pypi.org/project/llm-rates/
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=9.0.3; extra == "dev"
Provides-Extra: refresh
Requires-Dist: anthropic<2,>=0.122.0; extra == "refresh"
Dynamic: license-file

# llm-rates

Named `llm-rates` (not `llm-prices`, its name until 2026-08-21) because
PyPI's similarity check rejected `llm-prices` as too close to the existing
`llmprices` project — collapsed-separator collision, not an exact-name
clash. `llm-rates` was verified free in every form (`llm-rates`,
`llmrates`, `llm_rates`, singular `llm-rate`/`llmrate`) before adopting it;
don't re-litigate the name.

Source is maintained in a private monorepo. Releases are built and published by GitHub Actions through a PyPI Trusted Publisher; provenance attestations are visible on the PyPI file listing.

One vendored model-price catalog + a two-layer overlay (a packaged public
layer plus an optional, out-of-wheel private layer) + a shared Python
lookup/cost-math module, replacing four independently hand-maintained price
tables that were drifting apart. Full origin story and decisions: this
repo's sibling checkout `plugin/docs/plans/model-catalog-consolidation.md`
(cross-repo — `plugin` is a separate git repository from `libs`, so this is
a path reference, not a clickable link).

## Why this exists

Four tables held the same vendor facts (`plugin/scripts/prices/anthropic.json`,
`plugin/skills/tokenator/scripts/pricing.json`,
`sessions/src/sessmon/pricing.py`,
`factory-bench/runs/model-bench/model-bench-runner.py`). One of them silently
returned `$0.00` for any model it didn't recognise — a lookup bug, not a data
bug — and under-reported real spend by over $800 in one 30-day window. This
package fixes the *lookup logic* (never $0 for an unknown model) and gives
the data one home.

## Files

- `catalog.json` — vendored snapshot of
  [litellm's `model_prices_and_context_window.json`](https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json),
  filtered to the providers/models the portfolio actually uses. Vendor list
  price, per-single-token USD (litellm's native unit). Never hand-edit —
  regenerate prices via `refresh.py --apply`; for Anthropic-provider rows,
  the capability fields (`max_input_tokens`/`max_output_tokens`) are instead
  kept in sync with Anthropic's own live Models API via
  `refresh_context.py --apply` (litellm's snapshot is a third-party guess
  for those fields, not authoritative).
- `overlay.json` — the **public** overlay layer, packaged inside the wheel:
  family-rate fallbacks (`claude-opus` → tier rate, so an unrecognised new
  model prices at its family's rate instead of $0), model-ID aliases (dated
  suffixes → canonical key), retirement/deprecation history (a retired
  model's row is kept, never deleted, so historical transcripts still
  reprice correctly), `context_overrides` for the rare model whose real
  default-served context genuinely differs from its published maximum
  (empty as of 2026-08-21 — see "Context-window data" below), and the one
  public free-preview `actual_billing_overrides` row (`stealth/ox-alpha`).
  It's enough on its own — `load_overlay()` with no private overlay
  configured is a fully supported state. Portfolio-internal price facts
  (free-tier billing keys, per-consumer pin lists) live in an optional
  **private** overlay layer, `portfolio/overlay.json` — outside the wheel,
  resolved at call time via `LLM_RATES_OVERLAY`/`PORTFOLIO_DATA_ROOT`, and
  merged in by `load_overlay()`. See `portfolio/README.md` and the "Overlay:
  public and private layers" section below.
- `llm_rates/` — `lookup(model_id) -> PriceRecord` + cost math (in
  `__init__.py`); `catalog.json` and `overlay.json` ship inside this
  directory as package data so an installed wheel carries them alongside
  the code. Shared by every Python consumer; the sole PowerShell consumer
  (`tokenator.ps1`) can't import Python, so it reads a **generated** JSON
  table instead — see `generate_tokenator_table.py` below and the plan
  doc's "Known constraint".
- `refresh.py` — re-pulls the live litellm catalog, re-filters it to the
  same model set, diffs against the vendored `catalog.json`, and prints a
  report. Never applies silently — pass `--apply` to write. Price source
  only; does not touch capability fields for Anthropic-provider rows.
- `refresh_context.py` — re-pulls `max_input_tokens`/`max_tokens` for every
  Anthropic-provider `catalog.json` row from Anthropic's own live Models API
  (`client.models.retrieve()`), diffs against the vendored values, and
  prints a report. Never applies silently — pass `--apply` to write.
  Requires the `anthropic` package (`pip install "llm-rates[refresh]"`) and
  a resolvable Anthropic credential (`ANTHROPIC_API_KEY`,
  `ANTHROPIC_AUTH_TOKEN`, or an `ant auth login` profile) — never falls back
  to a guessed value if no credential resolves. Run manually; not wired
  into any scheduled job. `lookup()`/`cost()` never import this module or
  touch the network — see "Context-window data" below.
- `generate_tokenator_table.py` — emits a plain JSON price table shaped for
  `tokenator.ps1` (a `models` map of `{input, output, context}` per model id,
  plus flat `cache_read_multiplier`/`cache_write_multiplier` and a
  `default` row) from `catalog.json` + `overlay.json`, so that repo can hold
  a generated copy instead of a hand-maintained one. Deterministic — the
  same inputs always produce byte-identical output.

## Usage

```python
from llm_rates import lookup, cost, UnknownModelError

record = lookup("claude-sonnet-5")
record.input          # 2.0   ($/MTok)
record.output          # 10.0  ($/MTok)
record.source           # "catalog"
record.context           # 1000000 (context-window token limit, or None)

# A client-side "[1m]" resolvedModel signal (not a vendor id — see
# overlay.json's aliases) resolves to the same price and, as of 2026-08-21,
# the same context — the base id already reports the model's real
# 1,000,000-token window:
lookup("claude-sonnet-5[1m]").context   # 1000000

# One-shot cost for a turn:
usd = cost(
    "claude-opus-4-8",
    input_tokens=12_000,
    output_tokens=800,
    cache_read_tokens=50_000,
)

# An unrecognised model still resolves at its family rate:
lookup("claude-opus-4-9").source   # "overlay-family-fallback"

# A genuinely unknown vendor/model raises instead of returning $0:
try:
    lookup("some-new-vendor/mystery-model")
except UnknownModelError as e:
    ...
```

`lookup()` and `cost()` both accept optional `catalog=`/`overlay=` kwargs
(already-loaded dicts) — useful for tests, or for a caller that wants to load
once and reuse across many lookups instead of re-reading the JSON files each
call.

## Overlay: public and private layers

`overlay.json` (this directory) is the **public** layer and ships inside
the wheel — aliases, family fallbacks, retirements, `context_overrides`,
and the one public free-preview `actual_billing_overrides` row. It resolves
every current model family on its own; `pip install llm-rates` with no
further setup is a fully supported state, never a degraded one.

An optional **private** layer, `portfolio/overlay.json` (this repo only,
not packaged — see `portfolio/README.md`), carries portfolio-internal price
facts: free-tier billing overrides and per-consumer pin lists. `load_overlay()`
(no args) merges the two, private winning per key, once one of two
environment variables points at the private file:

1. `LLM_RATES_OVERLAY` — set to the private overlay's exact path. Must
   exist and parse if set and non-empty, else `OverlayNotFoundError` (a
   typo'd path silently falling back to public-only would recreate the
   silent-mispricing bug this package exists to kill). Set to `""` to mean
   "explicitly none".
2. `PORTFOLIO_DATA_ROOT` — if `LLM_RATES_OVERLAY` is unset, used to look
   for `<root>/config/llm-rates/overlay.json`; used if present, silently
   skipped if not.

With neither variable set, `load_overlay()` returns the public overlay
alone. `load_overlay(path)` with an explicit path always loads exactly that
one file, unmerged — today's meaning, unchanged.

## Refreshing the catalog

```bash
python refresh.py            # pull live litellm catalog, diff, print report
python refresh.py --apply    # also overwrite catalog.json with the diff
```

`refresh.py` always pulls the **live** GitHub-hosted catalog URL, never the
`litellm` pip package's bundled snapshot — that bundled copy is stale (it was
missing Haiku 4.5, Opus 5, Opus 4.8, Fable 5, and Sonnet 5 entirely as of
2026-07-28; see the plan doc's "Two traps found").

## Refreshing context-window data

```bash
python refresh_context.py            # pull the live Models API, diff, print report
python refresh_context.py --apply    # also overwrite catalog.json with the diff
```

Context-window limits (`max_input_tokens`) and output caps
(`max_output_tokens`) for Anthropic-provider rows come from Anthropic's own
**Models API** (`client.models.retrieve(model_id)`), not litellm's snapshot
— litellm is a third-party-maintained guess for these fields, and it was
wrong at least once (see "Pricing notes worth knowing" below). `lookup()`
itself never calls this API: it's a refresh-time-only script, exactly like
`refresh.py`, that writes into the vendored `catalog.json`, so
`lookup()`/`cost()` stay pure and offline (they price historical
transcripts and run behind a PowerShell overlay with no network access).
Requires `pip install "llm-rates[refresh]"` (the `anthropic` SDK) and a
credential the SDK's own resolution chain can find
(`ANTHROPIC_API_KEY`/`ANTHROPIC_AUTH_TOKEN`/`ant auth login`) — with none
resolvable, the script fails with an actionable message and writes nothing,
never a guessed value.

## Generating tokenator's pricing table

```bash
python generate_tokenator_table.py            # print to stdout
python generate_tokenator_table.py --out FILE  # write to FILE
```

This is a read-only export — it never touches `plugin`'s checkout. The
consumer is `plugin/tools/tokenator/pricing.json` (tokenator was revived as a
plugin tool 2026-09-05, plugin#1006, after living at
`plugin/skills/tokenator/scripts/` as a skill). That file is the
generated table as of 2026-09-05 — regenerate with `--out` pointed at it
whenever `catalog.json`/`overlay.json` change. `context` is passed through
from `lookup()` unchanged — the catalog value `refresh_context.py` verifies
against the Models API. A same-day override to 200,000 for bare
`claude-fable-5`/`claude-sonnet-5` was reverted 2026-09-05 after a scan of
every local transcript showed those sessions running past 500k; the correct
fix for a wrong window is always `catalog.json` via `refresh_context.py`. Before emitting anything, the generator proves (via
`verify_cache_multipliers()`) that its flat cache multipliers (0.1x read,
2.0x write of input — tuned to Claude Code's 1h-TTL cache behaviour) match
this package's real per-model cache rates for every model tokenator prices;
a divergence raises instead of silently drifting.

## Pricing notes worth knowing

- **`claude-sonnet-5` is $2.00/$10.00 per MTok, permanently** — not a
  time-limited introductory rate. It was announced as an introductory price
  through 2026-08-31, but the vendor cancelled the scheduled 2026-09-01 rise
  to $3.00/$15.00 (~2026-08-17). See `overlay.json`'s `notes`.
- **Groq OSS models list at $0.075–$0.29/MTok but bill $0.00** on the
  portfolio's free-tier key — the vendor list price lives in `catalog.json`,
  the actual billed rate lives in `portfolio/overlay.json`'s (the private
  layer's) `actual_billing_overrides`, picked up automatically once
  `LLM_RATES_OVERLAY`/`PORTFOLIO_DATA_ROOT` points at it. Both prices are
  correct; they answer different questions. Without the private overlay
  configured, these models resolve at the vendor's list price via
  `catalog.json` — never a silent $0 either way.
- **A retired/de-listed model keeps its row.** `claude-opus-4-1` (retired
  2026-08-05) and the Groq-delisted `llama-4-scout`/`qwen3-32b` entries stay
  priced at their historical rate so old transcripts still reprice
  correctly — deleting a row would silently re-price past usage at whatever
  family-fallback rate happens to apply now.
- **`claude-fable-5`, `claude-opus-5`, and `claude-sonnet-5` all serve
  1,000,000 tokens of context, natively** — confirmed by Anthropic's own
  Models API. A prior `overlay.json` `context_overrides` block forced all
  three to 200,000 (on the mistaken theory that catalog.json's
  `max_input_tokens=1000000` was an extended-context-beta-only ceiling);
  that override has been removed (corrected 2026-08-21, C8). Only
  `claude-haiku-4-5` among current Claude models genuinely has a
  200,000-token window.
- **`claude-fable-5-1` reads cache at $0.25/MTok (0.025× input), not the
  0.1× every earlier Anthropic model uses.** Its catalog row carries the
  explicit `cache_read_input_token_cost`; `lookup()` prefers a row's
  explicit rate over the derived multiplier, so the $1.00 the
  `claude-fable` family fallback would derive never applies. Context
  1,000,000, max output 128,000, same as Fable 5.

## Setup for consumers

```bash
pip install llm-rates
```

Same convention as `libs/plan-doc` and `libs/triad-base` — see
`plugin/CLAUDE.md`.

## Status

**Rollout step 1 of `model-catalog-consolidation.md` — the library itself,
built and tested.** Consumer migrations (`activity_cost.py`, `sessmon`,
`model-bench-runner.py`, `tokenator.ps1`) are steps 2–5, tracked separately
in the plan doc and out of scope for this package's initial PR.

## Testing

```bash
pytest llm-rates/tests/ -v
```
