Metadata-Version: 2.4
Name: tensorcost
Version: 0.4.0
Summary: One-line wrapper SDK for TensorCost — LLM observability and applied-mode inference routing for OpenAI and Anthropic clients.
Project-URL: Homepage, https://tensorcost.com
Project-URL: Documentation, https://docs.tensorcost.com
Project-URL: Source, https://github.com/vaadhlabs/tensorcost/tree/main/packages/sdk-python
Project-URL: Changelog, https://github.com/vaadhlabs/tensorcost/blob/main/packages/sdk-python/CHANGELOG.md
Project-URL: Bug Tracker, https://github.com/vaadhlabs/tensorcost/issues
Author-email: Vaadh Labs / TensorCost <engineering@tensorcost.com>
License: Apache-2.0
License-File: LICENSE
Keywords: anthropic,cost,llm,observability,openai,tensorcost
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: httpx<1.0,>=0.25
Provides-Extra: dev
Requires-Dist: pytest-mock>=3.10; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# tensorcost — Python SDK

One-line wrapper SDK for [TensorCost](https://tensorcost.com). Adds
fire-and-forget cost observability to any OpenAI or Anthropic Python
client without changing how your code calls the underlying provider.

## Install

```bash
pip install tensorcost
```

Runtime dependencies: `httpx` only.

## One-line example

```python
from openai import OpenAI
from tensorcost import wrap

client = wrap(OpenAI(api_key="sk-..."))

# use `client` exactly like a normal OpenAI client.
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hello"}],
)
```

Anthropic works identically:

```python
from anthropic import Anthropic
from tensorcost import wrap

client = wrap(Anthropic(api_key="sk-ant-..."))
client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=128,
    messages=[{"role": "user", "content": "hello"}],
)
```

## Configuration

`wrap()` resolves config in this order:

1. Explicit kwargs to `wrap(client, api_key=..., base_url=..., tenant_id=...)`
2. Environment variables: `TENSORCOST_API_KEY`, `TENSORCOST_BASE_URL`,
   `TENSORCOST_TENANT_ID`, `TENSORCOST_PROXY_URL`
3. Defaults: `base_url` defaults to `https://api.tensorcost.com`

If no `api_key` is found, `wrap()` raises `MissingConfigError`. Same if
`applied_mode=True` and no `proxy_url` is available from either source.
Both are the only failure modes the SDK does NOT swallow.

## Applied mode (Layer 2)

By default the SDK is observe-only: it fires telemetry after your call
completes, and your call goes directly to OpenAI / Anthropic. Applied
mode routes `chat.completions` (and Anthropic `messages.create`) through
the TensorCost inference-proxy, which can redirect calls to a different
provider based on per-tenant policy.

```python
from openai import OpenAI
from tensorcost import wrap

client = wrap(
    OpenAI(api_key="sk-..."),
    api_key="tc-...",              # TensorCost API key
    applied_mode=True,
    proxy_url="https://api.my-instance.tensorcost.com",
    # or set TENSORCOST_PROXY_URL in the environment
)

# Use client exactly as before — the proxy is transparent.
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hello"}],
)
```

The SDK forwards each provider's native request body unchanged. OpenAI
calls go to the proxy as OpenAI-shaped JSON; Anthropic calls go as
Anthropic-shaped JSON. The proxy identifies the source from the
`x-tc-provider-url` base URL and handles any translation internally.
This means the proxy response also comes back in the upstream provider's
native shape — your existing parsing code sees no difference.

The proxy decides per-request whether to route or pass through. If the
proxy call fails for any reason the SDK falls back to the direct provider
call and logs a warning — the customer's call always completes.

Telemetry (observe-mode) still fires on every call regardless of
`applied_mode`. The two paths are independent.

For the proxy design see
`docs/features/inference-proxy-applied-mode.md`.

## Applied-mode hardening (v0.4.0)

When `applied_mode=True`, you can configure retry policy, timeouts, lifecycle
hooks, and the fail-open circuit breaker.

### Retries

```python
from tensorcost import wrap, RetryConfig

client = wrap(
    OpenAI(api_key="sk-..."),
    applied_mode=True,
    proxy_url="https://...",
    retry=RetryConfig(max_attempts=5, base_delay_ms=500.0, max_delay_ms=30_000.0),
)
```

5xx responses, network errors, and 429 are retried with jittered exponential
backoff. A `Retry-After` header on 429 is honoured. 4xx errors (except 429)
are never retried.

### Timeouts

```python
client = wrap(
    OpenAI(api_key="sk-..."),
    applied_mode=True,
    proxy_url="https://...",
    timeout_s=30.0,   # default 60.0
)
```

Surfaces as `TensorCostTimeoutError` when exceeded.

### Lifecycle telemetry hooks

```python
from tensorcost import wrap, LifecycleEvent

def on_event(event: LifecycleEvent) -> None:
    # event.kind is one of:
    #   "before_request" | "after_response" | "on_retry"
    #   | "on_error" | "on_fallback"
    print(event.kind, event.model, event.attempt_number)

client = wrap(
    OpenAI(api_key="sk-..."),
    applied_mode=True,
    proxy_url="https://...",
    on_lifecycle_event=on_event,
)
```

Events contain only request metadata — never prompt content, response bodies,
or credentials. Errors raised by the callback are swallowed.

### Typed errors

Every failure from the proxy path is a subclass of `TensorCostError`:

```python
from tensorcost import (
    TensorCostError,
    TensorCostNetworkError,
    TensorCostTimeoutError,
    TensorCostProxyError,
    TensorCostQuotaError,
    TensorCostProviderError,
)

try:
    resp = client.chat.completions.create(model="gpt-4o-mini", messages=[])
except TensorCostQuotaError as e:
    print("rate limited; retry after", e.retry_after_ms, "ms")
except TensorCostError as e:
    print("tensorcost error", e.status, e.attempt)
```

### Circuit breaker

After 3 consecutive proxy 5xx responses the circuit opens and requests route
directly to the provider (fail-open). The circuit closes after 5 consecutive
probe successes.

Set `fail_open_enabled=False` if direct provider access is unavailable (e.g.
in-VPC deployments) — proxy failures then surface as `TensorCostProxyError`
instead of falling back.

```python
client = wrap(
    OpenAI(api_key="sk-..."),
    applied_mode=True,
    proxy_url="https://...",
    fail_open_enabled=False,   # raise on proxy error instead of falling through
)
```

## Fail-open guarantee

If TensorCost is unreachable, slow, or returns an error, your underlying
provider call still completes normally. The SDK logs a warning via
`logging.getLogger("tensorcost")` and moves on. We will never break your
production traffic.

## What gets sent

For every intercepted call we POST a JSON document with:

- SDK version, provider name, model, operation
- Request and response timestamps
- `prompt_tokens` / `completion_tokens` (or Anthropic equivalents) from
  the provider's `usage` field
- A correlation UUID
- The status (`success` / `error`) and an error message if the call
  failed

We do NOT capture prompt content, completion content, or any user data.
Cost calculation happens server-side.

Authentication uses a short-lived JWT obtained via
`POST /api/inference-proxy/sdk-token/exchange`. The long-lived API key never
leaves the SDK process after the first exchange and is cached in memory.

## Currently supported

- OpenAI (`openai >= 1.0`) — `chat.completions.create`,
  `completions.create`
- Anthropic (`anthropic >= 0.20`) — `messages.create`

## Coming soon

- AWS Bedrock (on the roadmap)
- Azure OpenAI (on the roadmap)
- Google Vertex AI (v1.0)
- Streaming responses (v1.0)
- Tool / function calling (v1.0)
- Async clients (`AsyncOpenAI`, `AsyncAnthropic`) (v1.0)
- LangChain / LlamaIndex adapters (v1.0)

## Development

```bash
pip install -e '.[dev]'
pytest
```

## License

Apache-2.0. See [LICENSE](LICENSE) for the full text.
