Metadata-Version: 2.4
Name: harness-token-trace
Version: 0.1.0
Summary: Per-API-call token cost decomposition for Claude Code, Claude desktop agents, and Cursor.
Author: Gregory Provo
License: MIT License
        
        Copyright (c) 2026 Gregory Provo
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/Provo17/tokentrace
Project-URL: Issues, https://github.com/Provo17/tokentrace/issues
Keywords: claude,claude-code,cursor,tokens,cost,observability,llm
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tzdata; sys_platform == "win32"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov>=4; extra == "dev"
Dynamic: license-file

# tokentrace

**Where your AI coding budget actually goes — measured, not estimated.**

tokentrace reads the transcripts your AI coding tools already write to disk and
decomposes every billable API call into its smallest cost-bearing components,
priced at real per-model rates. It answers questions your provider dashboard
can't: *which prompt cost $4.12?* *why is 57% of my bill cache reads?* *which
repo is expensive?*

Everything is derived from logged token counts. Nothing is sampled, projected,
or inferred.

```
COST BY COMPONENT  (measured per-category rates)
  fresh_in              412,880 tok        $2.06     0.4%
  cache_w5m           1,204,551 tok        $7.53     1.4%
  cache_w1h          17,441,092 tok      $174.41    32.4%
  cache_read        612,884,201 tok      $306.44    56.9%
  output              2,281,447 tok       $57.04    10.6%
```

That breakdown is the point. Most people assume they pay for answers; the data
usually says they pay for **memory** — re-reading conversation context on every
turn.

## What it reads

| Source | Where | What you get |
|---|---|---|
| `claude` | `~/.claude/projects/**/*.jsonl` | Claude Code, per API call |
| `claude-app` | Cowork / Claude desktop-app agent sessions (`audit.jsonl`, macOS) | per API call |
| `cursor` | tokscale's synced usage CSV | per event, Cursor's own reported cost |

All three merge into one report, each labelled. Claude sources carry a full
per-category dollar split; Cursor reports a lump cost per event, so tokentrace
shows its token mix instead of inventing a breakdown.

> **Not supported:** claude.ai chat (web/desktop chat pane) — note this is the
> *chat* side of the same desktop app whose Cowork agent sessions above *are*
> measured; the two write different things to disk. No local
> per-message token ledger exists for it — subscription chat is quota-gated
> rather than token-billed, so there is nothing on disk to measure.

## Install

```bash
pipx install harness-token-trace   # or: pip install --user harness-token-trace
```

The command is `tokentrace`; only the install name differs. PyPI refuses
`tokentrace` as too similar to the unrelated [`token-trace`][tt] project, so the
distribution is published under a distinct name while the command, the import
name, and this project all stay `tokentrace`.

[tt]: https://pypi.org/project/token-trace/

Python 3.9+, standard library only, no network calls.

(One exception, and only on Windows: `tzdata`, because Windows ships no IANA
timezone database for `zoneinfo` to read. It is that database as data — no code,
no transitive dependencies — and macOS and Linux still install nothing.)

## Connecting your harness

Two of the three sources need no setup at all — tokentrace reads transcripts the
tools already write. Only Cursor requires a step, because Cursor keeps its usage
data server-side.

| Harness | Setup | Read from |
|---|---|---|
| **Claude Code** | none | `~/.claude/projects/**/*.jsonl` |
| **Cowork** (Claude desktop app, agent sessions) | none on macOS; elsewhere set `TOKENTRACE_AGENT_SESSIONS` | `~/Library/Application Support/Claude/local-agent-mode-sessions/**/audit.jsonl` |
| **Cursor** | install and sync `tokscale` (below) | tokscale's `usage.csv` |

Run `tokentrace` and whichever of those exist on disk are picked up and labelled.
A source that finds nothing is skipped, not an error.

**Cursor:**

```bash
npm i -g tokscale
tokscale cursor login          # opens a browser; the token stays on your machine
tokscale cursor sync
```

Cursor's data is a point-in-time export, so re-run `tokscale cursor sync` to
refresh it — see [Keeping it fresh](#keeping-it-fresh) for automating that.

**Something missing?** If a harness you use isn't showing up, check the paths
above against your install and override them via the variables in
[Configuration](#configuration).

### A note on platforms 🍎

This tool was written on a Mac, by someone on a Mac, tested against transcripts
produced on a Mac. It shows.

Nothing here is *deliberately* macOS-only — it's stdlib Python and a handful of
paths — but macOS and Linux are the two platforms that have actually been run.
Here's the honest status:

| | Status |
|---|---|
| **macOS** | Home turf. All three sources, CI on every supported Python. |
| **Linux** | CI on every supported Python. Cowork needs `TOKENTRACE_AGENT_SESSIONS` pointed at your install. |
| **Windows** | Should work. Unproven. See below. 👀 |

**Windows: help wanted, and it might be a five-minute job.** The known platform
assumptions have been fixed — every file read and write pins UTF-8 instead of
inheriting cp1252, `--html --open` builds a real `file://` URL, and config and
state default to `%LOCALAPPDATA%` — so there's a decent chance it already runs.
There's a `windows-latest` CI job wired up and deliberately marked
non-blocking, so it reports without turning the build red for a platform the
maintainer can't reproduce on.

The one genuinely unknown piece is where Cowork keeps its agent sessions on
Windows. The default guesses `%APPDATA%\Claude\local-agent-mode-sessions` on the
theory that it sits beside the app's other state, but nobody has looked. If
you're on Windows, that's the single most useful thing you could tell us — and
if the guess is wrong, the fix is one line in
[`config.py`](src/tokentrace/config.py). Meanwhile you can just point at it:

```powershell
$env:TOKENTRACE_AGENT_SESSIONS = "C:\path\to\local-agent-mode-sessions"
```

Every path is overridable precisely so a wrong default is an inconvenience
rather than a wall. If you get it working — or get it *nearly* working — a PR or
an issue with the paths you found would be genuinely welcome. See
[Contributing](#contributing).

## Use

```bash
tokentrace                      # summary across all projects, by session
tokentrace --project myrepo     # filter project dirs by substring
tokentrace --session <id>       # one session, per-turn rows
tokentrace --tail 20            # last N calls
tokentrace --json               # machine-readable
tokentrace --html --open        # self-contained offline dashboard
```

Useful flags: `--no-cursor`, `--no-claude-app`, `--file <path.jsonl>`.

**Scoping rules.** `--project` filters Claude Code *and* Claude-app sessions (both
record a working directory) but excludes Cursor, whose usage data carries no
project attribution — tokentrace says so on stderr rather than silently reporting
a smaller number. `--file` names one Claude Code transcript, so the other sources
are out of scope by definition.

### The dashboard

`--html` writes one self-contained file (no CDN, no network) with daily spend,
cost by tool/model/component, your 20 priciest prompts, and a session drill-down
that groups calls under the user message that triggered them — the "what did
this question cost me" view. Every row deep-links, so you can paste a URL at an
exact turn.

Default output: `~/.local/state/tokentrace/dashboard.html`.

> The generated file embeds your real prompts, session names, and project paths.
> It is gitignored here for that reason. Treat it as private.

## Configuration

Every path the tool reads is overridable, so a non-standard install never leaves a
source silently contributing nothing.

| Variable | Purpose |
|---|---|
| `TOKENTRACE_TZ` | Display timezone, IANA name (e.g. `America/Chicago`). Defaults to system local. |
| `TOKENTRACE_PROJECTS_DIR` | Claude Code transcript directory. (`CLAUDE_PROJECTS_DIR` also works.) |
| `TOKENTRACE_CURSOR_CSV` | tokscale's synced Cursor usage CSV. |
| `TOKENTRACE_AGENT_SESSIONS` | Cowork / Claude desktop-app agent sessions. The default is the macOS path — set this to use the source off macOS. |
| `TOKENTRACE_LITELLM_URL` / `TOKENTRACE_OPENROUTER_URL` | Pin a rate source to a specific commit or mirror. |
| `XDG_CONFIG_HOME` / `XDG_STATE_HOME` | Standard; control where config and the dashboard live. |

For the sync script and LaunchAgents:

| Variable | Purpose |
|---|---|
| `TOKENTRACE_TOKSCALE_BIN` | Explicit path to `tokscale`. The script probes nvm, volta, asdf, fnm, mise, and Homebrew — set this for anything else (Nix, pnpm, a custom prefix). |
| `TOKENTRACE_SYNC_LOG` / `TOKENTRACE_SYNC_LOG_MAX_BYTES` | Sync log path and rotation threshold (default 1 MB). |
| `TOKENTRACE_DASHBOARD_INTERVAL` / `TOKENTRACE_SYNC_INTERVAL` | LaunchAgent intervals in seconds (defaults 900 / 1800), read by `install.sh`. |

Timezone matters more than it sounds: log timestamps are UTC, so bucketing by
UTC day puts evening work on the wrong day. tokentrace converts once, at load.

### Rates

Rates live in `src/tokentrace/rates.json` — data, not code, so refreshing them is
a reviewable diff.

**Keeping them current.** There is no Anthropic pricing API (`GET /v1/models`
returns context windows and capabilities, not rates), so rates come from two
independent public tables — [LiteLLM's](https://github.com/BerriAI/litellm) and
[OpenRouter's](https://openrouter.ai/api/v1/models). Both are unauthenticated;
no API key is needed and CI holds no secrets.

```bash
tokentrace --check-rates    # compare against upstream; exit 1 if they differ
tokentrace --update-rates   # apply changes both sources agree on
```

`--update-rates` writes wherever the refresh will actually survive: from a git
checkout it updates the tracked `src/tokentrace/rates.json` (the diff a
contributor reviews); from an installed package it writes
`~/.config/tokentrace/pricing.json` instead, because the in-package copy lives in
`site-packages`, is often not writable, and is erased by the next upgrade.

Neither runs on the normal path — `tokentrace` makes no network calls unless you
ask it to. A scheduled CI job runs `--check-rates` weekly and opens a PR when
upstream moves.

**A rate is applied automatically only when both sources agree.** Disagreement is
reported and nothing is written: for a tool pitched on measured numbers, two
independent sources concurring is evidence, and a conflict is exactly when a
human should look. Model keys are matched *exactly*, never by substring — both
sources list premium variants beside standard ones, and `claude-opus-4.8-fast`
bills at twice `claude-opus-4.8`.

**Rate changes append; they never overwrite.** When a rate moves, the outgoing
rate is kept as a dated period ending the day before the change, so a call is
always priced at the rate that applied *when it was made*. Both upstream sources
are point-in-time and forget yesterday's price — this history only exists here.
It also means upstream is never auto-applied to a model with an open dated
window (a promotional period, say): reconciling a point-in-time number against a
window is a judgement call.

**Overriding locally.** Drop a `~/.config/tokentrace/pricing.json` to correct or
extend the table without waiting for a release:

```json
{
  "claude-opus-4-8": { "in": 5.00, "out": 25.00, "read": 0.50, "w5m": 6.25, "w1h": 10.00 },
  "some-new-model":  { "in": 1.00, "out": 5.00,  "read": 0.10, "w5m": 1.25, "w1h": 2.00 }
}
```

An override wins outright, including over any built-in dated window for that
model.

**Unknown models are counted but not priced.** They contribute `$0` and are
labelled `UNPRICED` in the report, so a missing rate never reads as "free".

## Keeping it fresh

macOS (launchd) — regenerates every 15 min, syncs Cursor every 30 min:

```bash
./contrib/launchd/install.sh              # install + load
./contrib/launchd/install.sh uninstall

# intervals in seconds; the defaults are a starting point, not a recommendation
TOKENTRACE_DASHBOARD_INTERVAL=3600 ./contrib/launchd/install.sh
```

Linux — a cron pair does the same job:

```cron
*/15 * * * * tokentrace --html
*/30 * * * * /path/to/tokentrace/contrib/cursor-sync.sh
```

`cursor-sync.sh` classifies each run as `OK` / `AUTH-FAIL` / `FAIL` in its log,
so an expired Cursor token announces its own fix (`tokscale cursor login`)
instead of failing silently.

Optional shell shortcuts (`tcost`, `tcostcli`):

```bash
source /path/to/tokentrace/contrib/shell-functions.sh
```

## How the numbers are computed

**The billable unit is the `requestId`.** Claude Code writes one JSONL line per
content block and every line carries the *same* usage object. Summing lines
naively inflates cost by roughly 2.5×. tokentrace de-duplicates by `requestId`
before pricing anything. This is the single most common way homegrown token
scripts go wrong.

**Cache writes are not one rate.** Anthropic bills 5-minute cache writes at
1.25× input and 1-hour writes at **2× input**. Several popular tools flatten
both to the 5-minute rate, which understates real spend — in the author's own
data, 1-hour writes were ~32% of the total bill. tokentrace reads
`cache_creation.ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens`
separately and prices each correctly.

**Categories priced:**

| Category | Field | Rate |
|---|---|---|
| `fresh_in` | `input_tokens` | input |
| `cache_w5m` | `cache_creation.ephemeral_5m_input_tokens` | 1.25× input |
| `cache_w1h` | `cache_creation.ephemeral_1h_input_tokens` | 2× input |
| `cache_read` | `cache_read_input_tokens` | 0.1× input |
| `output` | `output_tokens` | output |
| `web_search` | `server_tool_use.web_search_requests` | $10 / 1k |

### Known limits

These are stated rather than papered over:

- **`cache_read` is a single scalar.** How much of it is system prompt vs tool
  definitions vs conversation history is *not* measurable from the logs — tool
  definitions never appear there. tokentrace will not estimate that split.
- **Cursor gives a lump cost per event**, with no per-category dollars and no
  session ID. Events are bucketed by day and labelled as Cursor-reported, and
  they cannot participate in `--project` filtering.
- **`claude-app` (Cowork) defaults to the macOS path.** The Windows default is an
  unverified guess and the Linux one doesn't exist; on either, set
  `TOKENTRACE_AGENT_SESSIONS` or the source is silently skipped.
- **The dashboard's `0600` permissions are POSIX-only.** On macOS and Linux the
  generated file is locked to your account. On Windows `chmod` only touches the
  read-only bit, so that guarantee does not hold — the file still contains your
  prompts, so treat it accordingly.
- **Unknown models price at `$0`** and are surfaced as such in the by-model
  table, so a missing rate looks like a missing rate.

## Development

```bash
git clone https://github.com/Provo17/tokentrace && cd tokentrace
pip install -e ".[dev]"
pytest
pytest --cov=tokentrace --cov-report=term-missing
```

Tests build synthetic transcripts in a `tmp_path` and redirect every filesystem
root, so the suite never reads your real transcripts or writes your real
dashboard. CI runs on Linux and macOS across Python 3.9–3.13.

The test suite is deliberately weighted toward the pricing and de-duplication
core rather than line coverage of the HTML/SVG builders. The properties worth
guarding are the ones that would silently produce a *wrong number*:

- one `requestId` bills once, no matter how many log lines it spans
- 1-hour cache writes bill at 2× input, never at the 5-minute rate
- legacy `cache_creation_input_tokens` lands in the 5m bucket, not the 1h one
- unknown models price at `$0` instead of falling back to a plausible rate
- day bucketing follows local time, not UTC
- Cursor's reported cost is passed through, never recomputed or split

Each of those is verified by a test that fails when the behaviour is mutated.

## Contributing

Issues and PRs welcome — especially additional harnesses (Copilot, Aider,
Windsurf), Linux packaging, and **anything that moves Windows from "should work"
to "does work"** (see [A note on platforms](#a-note-on-platforms-)). For Windows,
"I ran it and here's what broke" is a perfectly good contribution — so is "I ran
it and it was fine," which is currently unknown.

**Rate-table PRs are the best first contribution.** They're a small JSON diff in
`src/tokentrace/rates.json`, the test suite verifies the arithmetic, and the
weekly bot usually opens them for you pre-diffed — review the numbers against the
provider's published pricing and merge. Look for the `rates` label.

Rate corrections ship as immediate patch releases rather than waiting for a
feature batch: a stale rate makes the tool quietly wrong, which is worse than a
missing feature.

### Releasing

1. Bump `version` in `pyproject.toml` and move the entries under `## [Unreleased]`
   in [CHANGELOG.md](CHANGELOG.md) to the new version.
2. Tag and push: `git tag v0.1.1 && git push origin v0.1.1`.

The release workflow refuses to publish if the tag and the packaged version
disagree, runs the suite and a wheel smoke test, then publishes to PyPI via
Trusted Publishing (OIDC — no API token is stored in this repo) and opens a
GitHub release.

One-time PyPI setup before the first release: add a trusted publisher at
<https://pypi.org/manage/account/publishing/> for owner `Provo17`, repo
`tokentrace`, workflow `release.yml`, environment `pypi`.

The project rule: **every number must trace to a logged measurement.** Features
that estimate, extrapolate, or interpolate cost belong behind an explicit flag
and a visible label, if they belong here at all. New cost logic should come with
a test that fails if the arithmetic changes.

## License

MIT — see [LICENSE](LICENSE).
