Metadata-Version: 2.4
Name: tokenwatt
Version: 0.2.0
Summary: Know what your local LLM inference actually costs in electricity (Apple Silicon).
Author-email: Justin Stewart <justinstewart313+tokenwatt@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/mmmugh/tokenwatt
Project-URL: Repository, https://github.com/mmmugh/tokenwatt
Project-URL: Issues, https://github.com/mmmugh/tokenwatt/issues
Keywords: llm,inference,electricity,energy,cost,apple-silicon,mlx,proxy,openai
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: starlette<2,>=1.0
Requires-Dist: uvicorn>=0.30
Requires-Dist: httpx>=0.27
Requires-Dist: typer>=0.12
Requires-Dist: tiktoken>=0.7
Requires-Dist: zeus-apple-silicon>=1.0; sys_platform == "darwin" and platform_machine == "arm64"
Requires-Dist: pydantic>=2
Requires-Dist: pyyaml>=6
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Dynamic: license-file

# TokenWatt

[![CI](https://github.com/mmmugh/tokenwatt/actions/workflows/ci.yml/badge.svg)](https://github.com/mmmugh/tokenwatt/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
![Python](https://img.shields.io/badge/python-3.10--3.14-blue)
![Platform](https://img.shields.io/badge/platform-macOS_Apple_Silicon-lightgrey)

**Know what your local LLM inference actually costs you in electricity — per model, per request, on Apple Silicon. No sudo.**

TokenWatt is a transparent, OpenAI-compatible proxy. It sits in front of your local inference server, forwards every request byte-for-byte, measures the real per-rail SoC energy via Apple's IOReport (no password, ever), prices it at your utility rate, and logs a per-request, per-model electricity ledger.

```text
TokenWatt — electricity cost of local inference
  (numbers are ESTIMATED until you calibrate against a wall meter)
  last 24h :    119 req   0.0745 kWh   $0.0231

  model           type   req      kWh        $    J/tok   $/Mtok
  qwen3.6-27b     text   119   0.0745   $0.0231   5.019    0.432
```

*Real capture: an 84-minute agentic coding session (qwen3.6-27B building a Scheme interpreter,
119 streamed tool-calling turns) measured through TokenWatt on an M3 Ultra at $0.31/kWh.*

## The number people actually want

```text
$ tokenwatt compare
local electricity vs cloud LIST price for the SAME tokens (cloud snapshot 2026-06; list
price incl. their compute+margin, not just power; edit cloud.py):
  qwen3.6-27b   $0.0231 electricity — ~27.2× cheaper than gemini-2.5-flash-lite ($0.6286 vs $0.0231)
```

That session re-sent its growing context every turn — **6.07M input tokens for 53K output**, the
shape of high-context agentic coding. Locally, re-processing that context costs only electricity,
not a per-token API charge. A cloud API bills those re-sent tokens too — and even prompt caching
(which discounts re-sent context) only narrows it: local stays well ahead in this regime.

That leverage is specific to high-context agentic loops. For **balanced chat** (input ≈ output),
the cheapest cloud models are now *competitive* — gemini-2.5-flash-lite's $0.40/Mtok output is about
the same as local's $0.432, so there the win is privacy and control, not raw cost. TokenWatt shows
you which regime you're actually in — honestly, in both directions.

> Numbers are labeled **estimated (±15–30%)** until you calibrate against a wall meter, and the
> local figure is electricity only — it doesn't amortize the Mac. The point isn't one headline
> multiple; it's that you can see your real number, honestly, for *your* machine and *your* rate.

## Install

Not yet on PyPI — install from source:

```bash
uv tool install git+https://github.com/mmmugh/tokenwatt     # or, from a clone: uv tool install .
```

## Use

```bash
tokenwatt init                         # scaffold a commented tokenwatt.yaml (routes + your $/kWh)
tokenwatt serve -c tokenwatt.yaml      # one port in front of your local backends; no API key, no sudo
# point your OpenAI client (Pi / OpenClaw / Claude Code / any SDK) at http://127.0.0.1:7000/v1

tokenwatt report                       # today/month $, per-model $/Mtok and J/token
tokenwatt compare                      # your electricity vs named cloud prices, for the same tokens
tokenwatt wrap                         # a shareable "my inference bill" card
```

A single `--upstream` shortcut works too, with no config file:

```bash
tokenwatt serve --upstream http://127.0.0.1:8080 --rate 0.31
```

## What it does

For each request it: forwards byte-exact to your local server (`mlx-openai-server`, `mlx-vlm`,
Ollama, LM Studio, llama.cpp…) so response bodies are byte-identical to hitting the backend
directly; brackets
the request with Apple IOReport per-rail SoC energy (sudoless); subtracts a rolling idle baseline;
books model-load (cold-start) energy to a separate row; classifies the request type (text / vision /
embedding) and counts tokens from the backend's own usage when available; prices it at your flat or
time-of-use rate; and logs a per-request row — with a `request_id` that ties each cost row to a
structured JSONL operations log.

It is a *meter*, not a gateway: no API key, no rewriting, no buffering. Streaming, tool calls,
sampling params, chat templates, and `response_format` all pass straight through.

## Honesty

- Costs read **estimated (±15–30%)** until you run the wall-meter calibration.
- A model with no rate set shows `—`, never a fabricated `$0`; a real sub-cent cost shows `<$0.0001`,
  never a fake zero.
- `$/Mtok` for text uses completion tokens **including** reasoning/`<think>` tokens; the cloud
  comparison uses total input+output cost at dated list prices.

See the design spec and milestone plans under `docs/design/`.
