Metadata-Version: 2.5
Name: ai-tally
Version: 0.0.3
Summary: ai-tally Python SDK: cost-and-value observability for AI products (OTel gen_ai.* native)
Project-URL: Homepage, https://ai-tally.com
Project-URL: Documentation, https://ai-tally.com/docs/connect/python-sdk
Project-URL: Changelog, https://ai-tally.com/docs/changelog
Author: Tallynomics LLC
License-Expression: MIT
License-File: LICENSE
Keywords: ai,cost,finops,llm,observability,opentelemetry
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: jsonschema>=4.26.0; extra == 'dev'
Requires-Dist: pytest>=9.1.1; extra == 'dev'
Requires-Dist: pyyaml>=6.0.3; extra == 'dev'
Requires-Dist: ruff<0.18,>=0.17.2; extra == 'dev'
Provides-Extra: litellm
Requires-Dist: litellm==1.105.1; extra == 'litellm'
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == 'mcp'
Requires-Dist: pyyaml>=6.0.3; extra == 'mcp'
Description-Content-Type: text/markdown

# ai-tally (Python SDK)

The Python SDK for [AI Tally](https://ai-tally.com). It records what each AI call in your app
costs, which feature made the call, and which of your customers it was for. By default it sends
only counts, costs and hashed ids. Your prompts and answers stay in your app.

```bash
pip install ai-tally
```

The SDK has no dependencies, and recording never throws an error into your code. If something
inside it fails, it logs the problem and your call goes ahead as normal.

Full documentation: https://ai-tally.com/docs/connect/python-sdk

## Quick start

Call `aitally.init()` once when your app starts. Pass the ingest key from the dashboard's Setup
page, or set it as `TALLY_KEY`.

```python
import aitally

aitally.init("tally_sk_live_...")

# Your existing calls are now recorded. You don't change them:
client.chat.completions.create(model="gpt-4o-mini", messages=[...])     # openai
client.messages.create(model="claude-sonnet-4-5", messages=[...])       # anthropic
client.models.generate_content(model="gemini-2.5-flash", contents=...)  # google-genai
bedrock.converse(modelId="us.amazon.nova-pro-v1:0", messages=[...])     # boto3
```

`init()` is safe to call more than once, and it doesn't wait on the network. If the key is missing
or the address is refused, it logs one warning and records nothing. If the key is wrong or AI Tally
is down, your calls still run normally and the SDK logs a warning.

### What gets recorded on its own

| Library | Calls | Async | Streaming |
| --- | --- | --- | --- |
| `openai` | `chat.completions.create`, `responses.create`, `embeddings.create` | yes | yes |
| `anthropic` | `messages.create`, `messages.stream` | yes | yes |
| `google-genai` | `models.generate_content`, `models.generate_content_stream` | yes (`client.aio`) | yes |
| `boto3` (Bedrock) | `converse`, `converse_stream` | no | yes |

The SDK only touches a library you already have installed. It doesn't install any of them.

Anything else needs one line after the call, `aitally.record_llm_call(...)`. That includes
Bedrock `invoke_model`, the old `google-generativeai` package, and any other client. For LiteLLM,
see the LiteLLM section below.

Good to know:

- **Choose which libraries to record.** `aitally.init(key, instrument={"openai", "anthropic"})`
  records only those two. The names are `"openai"`, `"anthropic"`, `"google"` and `"bedrock"`.
  `instrument=False` turns all of them off.
- **Other providers through the `openai` library.** Calls to Groq, Fireworks, Together, DeepSeek,
  xAI, OpenRouter, Mistral, Azure or Gemini through the `openai` library are labeled with the real
  provider, never as OpenAI. A provider AI Tally has no price for shows a blank cost, not a guess.
  A host the SDK doesn't recognize is labeled `openai-compatible` and its cost is blank.
- **Streaming with OpenAI.** OpenAI only reports token counts on a stream if you ask for them with
  `stream_options={"include_usage": True}`. Without that, the cost shows as blank. Pass
  `aitally.init(..., instrument_stream_usage=True)` and the SDK asks for you.
- **Bedrock streams** are recorded once your code has read the stream to the end.
- **Bedrock inference profiles and provisioned models.** If the model can't be read from the ARN
  (an application inference profile or a provisioned model), the call is recorded with no model
  and a blank cost.
- **Other kinds of cost** are one line each: `aitally.record_vector_call` for vector search,
  `aitally.record_tool_call` for your own tool calls, and `aitally.record_embedding_call` for
  embeddings not made through a recorded client. They do nothing until `init()` has run.
- **Before your process exits**, the SDK sends what it has buffered. Call `aitally.flush()` to send
  it sooner. `aitally.uninstrument()` undoes everything, which helps in tests.
- **Calls through the AI Tally proxy** are recorded by the proxy, so the SDK skips them and nothing
  is counted twice.

### Remove manual lines you no longer need

If you added `aitally.record_llm_call(...)` after a Gemini or Bedrock `converse` call, delete it.
`init()` records those calls now. Until you do, the SDK notices most duplicates (same call, same
thread or async task, a few seconds apart), drops the copy and warns you once. Don't rely on that:
it can't catch a manual line that runs in a different thread or task, for example under
`asyncio.gather`.

### The connection must be secure

The SDK sends your key with every request, so it only connects over `https://`. Plain `http://`
works only for your own machine (such as `localhost` or `127.0.0.1`), so local testing still works. To use
`http://` for anything else, such as another machine on your network, you have to say so:
`aitally.init(key, endpoint=..., allow_insecure_http=True)`, or `TALLY_ALLOW_INSECURE_HTTP=1`.

If the address is refused, the SDK logs one warning and records nothing. It also doesn't follow
redirects, so give it the final address.

## Cost per customer

Tell the SDK which of your customers a call is for, and AI Tally can show cost per customer. Set
the customer once, for example in your request middleware, and every call inside picks it up:

```python
import aitally

aitally.init("tally_sk_live_...")

with aitally.start_trace(feature_tag="support-bot"), aitally.with_account("acct_northwind"):
    client.chat.completions.create(model="gpt-4o", messages=[...])   # recorded for acct_northwind
    aitally.record_tool_call(provider="openai", tool="web_search")
```

A batch job that works through many customers can pass the customer on each call instead:

```python
import aitally
from aitally.pricing import Usage

with aitally.start_trace():
    for account in accounts:
        aitally.record_llm_call(
            provider="openai", model="gpt-4o",
            usage=Usage(input_tokens=1200, output_tokens=300),
            account_id=account,
        )
```

`with_account(None)` clears the customer for a block, for background work that shouldn't inherit
it.

**Your customer ids stay in your app.** The SDK replaces each id with a hash (HMAC-SHA256) made
with a key that belongs to your organization only. AI Tally sees the hash, never the id, and the
same id gives a different hash for every organization. The SDK first fetches this hashing key
when `init()` runs. Calls made before it arrives are recorded without a customer. If that first
fetch fails, calls are still recorded but without a customer until the app restarts.

You can add a readable name so the dashboard shows "Northwind Traders" instead of a short hash:

```python
with aitally.with_account("acct_northwind", label="Northwind Traders"):
    ...
```

The name is kept apart from your usage data. Names are optional.

### Hashing an id yourself

If you use the AI Tally proxy instead of the SDK, send the customer as a hash in the
`X-Tally-Account-Id-Hash` header. Make the hash on your machine with the same key:

```python
from aitally import hash_account
h = hash_account("acct_northwind")
```

```bash
python -m aitally.hash_account acct_northwind   # reads TALLY_KEY
```

Unlike recording, `hash_account` raises an error if it can't get the hashing key.

## Command line

The SDK comes with a `tally` command. It uses the same key as `aitally.init()`.

```bash
tally login                    # saves your key (not shown on screen) to ~/.config/tally/config.ini
tally health                   # what your key can do, and model calls in the last 24 hours
tally keys                     # your organization's keys, without the secrets (needs a write or admin key)
tally test-event               # sends one empty test event and shows what arrived
tally spend --month 2026-10    # that month's spend by feature and provider
```

`TALLY_KEY` overrides the saved key, so CI needs no config file. Every read command takes `--json`.
The key is never printed. A number AI Tally couldn't work out shows as `unknown`, never 0. If some
calls in a month have no price, the total shows as "at least".

## LiteLLM

If you call models through LiteLLM, install the extra and add the AI Tally logger:

```bash
pip install "ai-tally[litellm]"
```

```python
import litellm
import aitally
from aitally.integrations.litellm import TallyLiteLLMLogger

aitally.init("tally_sk_live_...", instrument=False)
litellm.callbacks = [TallyLiteLLMLogger()]

litellm.completion(
    model="gemini/gemini-2.5-flash",
    messages=[...],
    litellm_metadata={
        "tally_account": customer_id,          # hashed before it leaves your app
        "tally_feature": "title_search.main",  # shows as the feature
        "tally_session": session_id,           # optional
        "tally_account_label": "Northwind",    # optional, readable name
    },
)
```

- **Use `instrument=False`.** LiteLLM sometimes uses the `openai` library, so without it a call
  could be counted twice. With it, direct provider calls outside LiteLLM aren't recorded.
- **Use `litellm_metadata`, not `metadata`.** LiteLLM sends `metadata` on to the provider in some
  cases. It never sends `litellm_metadata`.
- **Every call needs `tally_account`.** Without it the cost counts toward your total but not toward
  any customer. `with_account` doesn't apply to LiteLLM calls.
- **Streams** need `stream_options={"include_usage": True}`, except for Anthropic and Gemini.
  Without it the cost is blank.
- **Shut down before you exit.** A normal script is fine. In an async script, await `ashutdown()`
  before `asyncio.run(main())` returns, or the last calls are lost:

  ```python
  from aitally.integrations.litellm import ashutdown, shutdown

  shutdown(timeout_s=5)         # batch jobs, after your event loop has finished
  await ashutdown(timeout_s=5)  # inside a running loop, such as a server's shutdown hook
  ```
- **Supported providers.** Calls that aren't streamed get token counts only for OpenAI,
  Anthropic, Gemini (the `gemini/` route, not Vertex) and Mistral. Non-streamed calls to other
  providers show a blank cost.
- The extra installs LiteLLM 1.105.1, the version the logger was tested with. On other versions it
  still runs, but token counts come out blank.

## Keeping replay samples on your own server

To try other models on your real traffic without sending your prompts to AI Tally, the SDK can save
a small sample of calls on your own server:

```python
aitally.init(replay_capture="local", replay_dir="/var/lib/tally/replay")
```

It saves only while an admin has turned on local capture for your organization. By default it
saves 1 in 100 calls that aren't streamed (`replay_sample_rate`) and keeps them for 7 days
(`replay_retention_days`). Nothing is sent to AI Tally. **The files hold real prompts and
answers**, so treat that folder like any other customer data.

The SDK only deletes its own sample files, in a folder it marked as its own. If an admin turns
local capture off, the SDK deletes the samples. To delete them yourself:

```bash
tally eval purge --samples /var/lib/tally/replay                      # everything
tally eval purge --samples /var/lib/tally/replay --before 2026-10-01  # older days only
```

Or from code: `aitally.purge_local_samples(replay_dir, before=...)`.

### Comparing models on those samples

```bash
pip install openai anthropic   # the providers you want to test
tally eval --samples /var/lib/tally/replay --models gpt-4o-mini,claude-haiku-5-5 --max-spend-usd 5
```

Each sample is run again on each model you list, using your own provider keys
(`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`). A judge model then compares each answer with the one your
app actually gave. You get one row per model with its cost, the judge's cost, how often it won
(with a 95% range), how often it refused, and how many samples were scored, skipped or failed.
Skipped and failed samples never count as losses.

The models you compare, and the judge, must be OpenAI or Anthropic models. Only samples from your
OpenAI and Anthropic calls are used. Samples that used tools or images are skipped.

- `--max-spend-usd` is a hard limit on what the run can spend. The command shows the worst case
  and asks before it starts. `--yes` skips the question.
- `--judge` picks the judge model. Otherwise it uses your organization's judge.
- Answers are capped at 1024 tokens (`--max-output-tokens`, `--judge-max-output-tokens`). An
  answer cut off at the cap counts as failed.
- `--format json` also checks that each answer is valid JSON. `--json` prints the results as JSON.

`tally eval` only runs while local capture is on. If your organization has turned it off, it also
deletes the samples.
`aitally.run_local_eval(...)` does the same from code.

## Setup help for your coding agent (MCP)

`aitally.init()` records model calls by itself. Vector search, tool calls and cost per customer
need a few lines in your code. The onboarding MCP server gives your coding agent (Claude Code,
Cursor) the right lines for your app. It runs on your machine, your code never leaves it, and it
only suggests code. It never edits your files.

```bash
pip install "ai-tally[mcp]"
```

This installs the `tally-onboarding-mcp` command. Give it your ingest key as `TALLY_KEY`. Only the
`coverage_report` tool uses the key. The others work offline.

### Claude Code

```bash
claude mcp add ai-tally-onboarding --env TALLY_KEY=tally_sk_... -- tally-onboarding-mcp
```

To share it with your team, add it to `.mcp.json` in your project. `${TALLY_KEY}` is read from each
person's own environment, so the key stays out of the file:

```json
{
  "mcpServers": {
    "ai-tally-onboarding": {
      "command": "tally-onboarding-mcp",
      "args": [],
      "env": { "TALLY_KEY": "${TALLY_KEY}" }
    }
  }
}
```

### Cursor

Add this to `~/.cursor/mcp.json`. A project `.cursor/mcp.json` works too, but never commit a file
that holds your key:

```json
{
  "mcpServers": {
    "ai-tally-onboarding": {
      "command": "tally-onboarding-mcp",
      "args": [],
      "env": { "TALLY_KEY": "tally_sk_..." }
    }
  }
}
```

If your agent can't find `tally-onboarding-mcp`, give the full path instead, for example
`"command": "/path/to/.venv/bin/tally-onboarding-mcp"`.

### The tools

| Tool | What it does |
|---|---|
| `detect_stack` | takes the contents of your `requirements.txt`, `pyproject.toml` or `package.json` and lists the providers, frameworks and vector databases it finds, what is already recorded, and what it has no recipe for |
| `get_recipe` | returns one setup recipe, by name (for example `pinecone` or `fastapi`) |
| `generate_startup` | writes the `aitally.init()` line for your app |
| `generate_middleware` | writes the middleware that sets the customer and feature, using the header or lookup you tell it |
| `instrument_call_site` | writes the `record_*` line for one call in your code |
| `explain_layer` | explains which `record_*` call covers a kind of cost, and why |
| `coverage_report` | asks AI Tally which kinds of cost are arriving, and why any are missing |

The server never guesses. It won't pick a customer id for you: tell it how your app knows the
customer, or the customer part stays empty. Anything it can't fill in from your code is left as a
visible `<FILL:...>` for you to complete. It also won't suggest a `record_llm_call` for a call
`init()` already records, since that would count the cost twice.

#### `coverage_report` settings

`coverage_report` is the only tool that connects to anything. It asks AI Tally which kinds of cost
are arriving for your organization.

| Variable | What it is |
|---|---|
| `TALLY_KEY` | your ingest key. The report only ever covers your own organization |
| `TALLY_GATEWAY_URL` | optional. The address to ask. Defaults to the same address the SDK uses. Must be `https://` unless it's your own machine |
| `TALLY_MCP_TENANT_ID` | optional. Your organization id (shown on the dashboard's connector setup). If set, a key for a different organization is refused |

The tool reads the key when it needs it and never saves, shows or logs it. If AI Tally can't be
reached, every kind of cost shows as `unknown` with the reason. A kind of cost only shows as
covered when real data has arrived. Pass `wired` with the kinds you just set up and the report
says "waiting for the first event" instead of "not set up".

### Node and TypeScript apps

The SDK is Python, but the MCP server also helps with Node apps. Give `detect_stack` your
`package.json` and it suggests setups that need no SDK:

| If your app uses | It sets up |
|---|---|
| `openai`, `@ai-sdk/openai` | the AI Tally proxy: `baseURL: "<ingest>/openai/v1"` and an `X-Tenant-Key` header holding your ingest key |
| `@anthropic-ai/sdk`, `@ai-sdk/anthropic` | the proxy: `baseURL: "<ingest>/anthropic"` (`<ingest>/anthropic/v1` for `@ai-sdk/anthropic`) and the same header |
| `@opentelemetry/*`, `@traceloop/*` | an OpenTelemetry exporter sending to `<ingest>/v1/otlp/traces` |
| any of the above | cost per customer, with a hashed customer id in the `X-Tally-Account-Id-Hash` header |

`<ingest>` is `https://ingest.ai-tally.com`. The proxy passes your provider key on to the provider
and doesn't store it, but the key does pass through AI Tally. OpenTelemetry never sends it. An
admin has to turn the proxy on first, under Settings > API keys. Until then it answers `403`.

As in Python, the customer id is hashed in your app and only the hash is sent. Use one setup per
call: if a call goes through the proxy and OpenTelemetry, it's counted twice, and `detect_stack`
warns you. There's no Node recipe yet for Gemini, Bedrock, Mistral, Cohere, Groq or Node vector
databases. `detect_stack` lists them as gaps.

## License

MIT. See `LICENSE`.
