Metadata-Version: 2.5
Name: tokli
Version: 0.1.0
Summary: Report-only usage tracking for OpenAI, Anthropic, Gemini, DeepSeek, xAI, Mistral, Qwen, GLM, Kimi and OpenRouter — wrap your client, we read the tokens.
Project-URL: Homepage, https://tokli.dev
Project-URL: Source, https://github.com/joagarc2/tokli-v2
Project-URL: Issues, https://github.com/joagarc2/tokli-v2/issues
Author-email: tokli <hello@tokli.dev>
License-Expression: MIT
License-File: LICENSE
Keywords: anthropic,cost,deepseek,gemini,glm,kimi,llm,mistral,openai,openrouter,qwen,tokli,usage,xai
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.27.0; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mypy>=1.14; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.25; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0.0; extra == 'gemini'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# tokli

Report-only usage tracking for [tokli](https://tokli.dev) — a cost & usage dashboard for AI
APIs. Wrap your existing provider client, keep calling it exactly as before, and tokli reports
the token usage in the background. **We never see your provider API key** and we never sit in
your request's critical path.

Supports OpenAI, Anthropic, Gemini, DeepSeek, xAI, Mistral, Qwen, GLM, Kimi and OpenRouter.

## Install

```bash
pip install tokli
```

`openai`, `anthropic` and `google-genai` are optional extras. tokli has **zero runtime
dependencies** and never pulls a provider SDK, or a version bound, into your app.

## Quickstart

```python
from openai import OpenAI
from tokli import wrap_openai

client = wrap_openai(OpenAI())  # ingest key from TOKLI_INGEST_KEY
client.chat.completions.create(model="gpt-5.4", messages=messages)
```

The returned client behaves exactly like the original: every attribute we don't instrument
passes straight through, `isinstance(client, OpenAI)` still holds, and
`client.with_options(...)` / `client.copy(...)` hand back clients that are **still**
instrumented.

Without an ingest key (neither `ingest_key` nor `TOKLI_INGEST_KEY`), the wrapper is a
**silent no-op**: your app keeps working exactly as before, nothing is reported.

### Async

The same function. There is no separate async wrapper — `wrap_openai(OpenAI())` and
`wrap_openai(AsyncOpenAI())` return the same kind of object.

```python
from openai import AsyncOpenAI
from tokli import wrap_openai

client = wrap_openai(AsyncOpenAI())
await client.chat.completions.create(model="gpt-5.4", messages=messages)
```

### Per-feature attribution

```python
checkout = client.with_feature("checkout")  # immutable and chainable
await checkout.chat.completions.create(...)  # every event tagged "checkout"
```

`with_feature` exists at runtime but type checkers cannot see it, because Python has no
intersection types and the wrappers return your client's own type so autocompletion survives.
For a type-checked equivalent, use the free function:

```python
from tokli import with_feature

checkout = with_feature(client, "checkout")
```

## Available wrappers

`wrap_openai`, `wrap_anthropic`, `wrap_gemini`, `wrap_deepseek`, `wrap_xai`, `wrap_mistral`,
`wrap_qwen`, `wrap_glm`, `wrap_kimi`, `wrap_openrouter` — all share the same
`(client, **options) -> client` signature and the `.with_feature(tag)` chaining above. The
seven OpenAI-compatible ones take the `openai` package pointed at the provider's base URL;
each wrapper's docstring carries the exact URL and the provider's caching quirks.

## Config

| Argument | Env var | Default | Notes |
| --- | --- | --- | --- |
| `ingest_key` | `TOKLI_INGEST_KEY` | — | Required. Without it, the wrapper is a no-op. |
| `endpoint` | `TOKLI_ENDPOINT` | `https://api.tokli.dev` | Ingest API base URL. |
| `timeout_ms` | `TOKLI_TIMEOUT_MS` | `2000` | Timeout for the report request. |
| `flush_ms` | `TOKLI_FLUSH_MS` | `2000` | How long to wait at exit for pending reports; `0` disables. |
| `on_error` | — | no-op | Called if reporting fails; never raises into your code. |

An argument beats the environment variable, which beats the default. An unusable value (a
typo in an env var) is skipped rather than raised — a misconfiguration must not take your app
down.

**`on_error` runs on a worker thread.** Keep it thread-safe and don't touch request-scoped
state from it. It receives `(error, reason)`, where reason is `"transport"`, `"no_usage"` or
`"parse"`.

**`flush_ms` is per-wrap but the worker pool is per-process.** If two clients are wrapped with
different values, the pool waits for the longest one — a short deadline must not discard
another wrapper's pending events. So `flush_ms=0` only disables the wait if it is the only
value configured in the process.

## What gets reported

We instrument the calls that bill tokens, and only those:

| Client | Reported |
| --- | --- |
| OpenAI & compatible (DeepSeek, xAI, Mistral, Qwen, GLM, Kimi, OpenRouter) | `chat.completions.create` / `.parse`, `responses.create` / `.parse` / `.compact` |
| Anthropic | `messages.create` / `.parse` |
| Gemini | `models.generate_content` / `.generate_content_stream`, and the same two under `aio.models` |

Streaming and non-streaming are both covered, as is `with_options()` / `copy()`, which re-wrap
the client they return.

### What is *not* reported

These pass through untouched. If you use one, its spend will **not** appear in tokli — this
list is the whole of it. For the namespaces we do instrument, a test enumerates every public
method and fails the moment a provider SDK grows one nobody has classified.

| Not instrumented | Why |
| --- | --- |
| `client.chats` (Gemini) | A stateful helper over `generate_content`. **The official quickstart uses it**, so it is the easiest one to trip over — call `models.generate_content` if you want it counted. |
| `chat.completions.stream()`, `responses.stream()`, `messages.stream()` | Helper wrappers with their own consumption surface. Use `create(stream=True)` if you want tokli to see it. |
| `responses.retrieve()` / `.cancel()` / `.delete()` | Idempotent or administrative — reporting `retrieve` would bill the same response once per poll. |
| `chat.completions.retrieve/list/update/delete`, `chat.completions.messages` | Manage stored completions; they bill no tokens. |
| `messages.count_tokens()`, `models.count_tokens()`, `models.compute_tokens()`, `responses.input_tokens` | Counting only; they bill nothing. |
| `messages.batches` (Anthropic) | A separate pipeline whose usage arrives out of band. |
| `responses.connect()` | Opens a realtime session; out of scope for v1. |
| `beta.*` namespaces (OpenAI and Anthropic) | Moving targets; they get instrumented once they stabilise. |
| `with_raw_response.*`, `with_streaming_response.*` | Return the raw HTTP exchange instead of a parsed body. |
| Gemini `models.embed_content`, `generate_images`, `generate_videos`, `edit_image`, `upscale_image`, `recontext_image`, `segment_image`, `list`, `get`, `update`, `delete` | They bill on non-token meters, or bill nothing. |

**Background responses.** `responses.create(background=True)` returns immediately with no
usage, so there is nothing to report at that point — the real numbers arrive later through
`responses.retrieve()`, which we deliberately leave uninstrumented (it is idempotent, and
reporting it would count the same response on every poll). A background call that also
streams **is** reported: its terminal event carries the usage. If you rely on non-streaming
background calls, report those yourself.

## Streaming

For streaming chat completions on OpenAI, DeepSeek and Qwen, pass
`stream_options={"include_usage": True}` so the final chunk includes token usage — without it
there is nothing for tokli to report. xAI, GLM, Kimi and OpenRouter send it either way.
**Mistral takes no `stream_options` at all** — its schema rejects unknown fields. Anthropic,
Gemini and the Responses API need no flag.

If you abandon a stream part-way (`break` out of the loop), nothing is reported: we cannot
know what you were billed, and we would rather report nothing than guess.

## How it works

Your code keeps calling the provider with your own key. After the response comes back, the SDK
reads the `usage` object already present in it and hands those raw numbers to a small
background worker pool — it never blocks your request, your event loop, or your key. Cost is
computed server-side against a versioned price table, never by the SDK.

Reporting can never break your app: if anything in our own path raises — including your
`on_error` handler — you still get the provider's response untouched.

A few consequences worth knowing:

- **The report POST reuses one keep-alive connection per worker thread.** A connection idle
  for more than 5 seconds is replaced rather than reused, and a failed send is dropped, never
  retried: a retry could duplicate an event if the server processed it before the socket
  broke, and a duplicate corrupts your cost far worse than a lost event does.
- **`http_proxy` / `https_proxy` are not read.** tokli talks to its own ingest endpoint over
  `http.client`, which ignores proxy environment variables — no surprise routing. Corporate
  proxy support is not in v1.
- **TLS verification is always on** and cannot be disabled.
- **At exit** the SDK waits up to `flush_ms` for pending reports, so short scripts and CLI
  jobs don't systematically lose their last event.
- **After `fork()`** (gunicorn/uvicorn `--preload`) the child drops the queue it inherited, so
  workers never resend the parent's backlog.

## Requirements

Python 3.10+.

## Links

- [tokli.dev](https://tokli.dev)
- [Source & issues](https://github.com/joagarc2/tokli-v2)
