Metadata-Version: 2.4
Name: keel-llm-otel
Version: 0.1.1
Summary: OpenTelemetry metrics + span events for keel-llm-reliability. A pure wrapper around ResilientClient — emits per-attempt metrics and span events from the visible-degradation trail. No coupling: keel-llm-reliability stays observability-free.
Project-URL: Homepage, https://github.com/keelplatform
Project-URL: Changelog, https://github.com/keelplatform/keel/blob/main/py/packages/llm-otel/CHANGELOG.md
Author: Raj Yakkali
License: MIT
Keywords: keel,llm,metrics,observability,opentelemetry,otel,tracing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: keel-llm-reliability>=0.1.2
Requires-Dist: opentelemetry-api>=1.20
Provides-Extra: auto
Requires-Dist: opentelemetry-instrumentation>=0.40b0; extra == 'auto'
Provides-Extra: full
Requires-Dist: httpx>=0.27; extra == 'full'
Requires-Dist: opentelemetry-instrumentation-httpx>=0.40b0; extra == 'full'
Requires-Dist: opentelemetry-instrumentation>=0.40b0; extra == 'full'
Provides-Extra: sdk
Requires-Dist: opentelemetry-sdk>=1.20; extra == 'sdk'
Provides-Extra: starter
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.20; extra == 'starter'
Requires-Dist: opentelemetry-instrumentation>=0.40b0; extra == 'starter'
Requires-Dist: opentelemetry-sdk>=1.20; extra == 'starter'
Description-Content-Type: text/markdown

# keel-llm-otel

> OpenTelemetry metrics + span events for [`keel-llm-reliability`](https://pypi.org/project/keel-llm-reliability/). A **pure external wrapper** around `ResilientClient` — emits signals from the visible-degradation trail *after* each call. `keel-llm-reliability` itself stays observability-free.

Part of the **Keel** toolkit. Composes cleanly: drop in over any `ResilientClient` instance, configure your OTel SDK however you like, see your multi-model LLM calls light up in Jaeger / Datadog / Honeycomb / Grafana / wherever.

## Is this for you?

**Adopt when** — you run `keel-llm-reliability` in production with OpenTelemetry already in your stack and want per-attempt metrics + span events flowing into your existing tracing/metrics backend without writing the bridge yourself.
**Skip when** — you're not on OTel; you're prototyping (the visible `Attempt` trail in return values is plenty for development); you want a different backend (write your own thin wrapper over `ResilientClient` — the pattern here is ~80 LOC).

## Install

```bash
pip install keel-llm-otel
# you bring your own OTel SDK + exporter, e.g.:
#   pip install opentelemetry-sdk opentelemetry-exporter-otlp
```

## Four seamless paths — pick one

### Path 0 — One-line setup (fastest trial) — `[starter]` extra

Production OTel SDK setup is famously verbose. `[starter]` bundles the SDK + an OTLP/gRPC exporter + a `setup()` helper that does the standard wiring in one line:

```bash
pip install 'keel-llm-otel[starter]'        # SDK + OTLP exporter + auto-instrumentation
```

```python
from keel_llm_otel.starter import setup
setup()    # SDK configured, OTLP export to localhost:4317, auto-instrumentor active.

# That's it. Your existing ResilientClient code emits metrics + span events.
```

Env-driven defaults (override with kwargs to `setup()`):
- `OTEL_EXPORTER_OTLP_ENDPOINT` → default `http://localhost:4317` (SigNoz / Jaeger / your collector)
- `OTEL_SERVICE_NAME` → default `keel-app`

> **Production at scale** typically configures the SDK its own way (custom resource attributes, sampling, batching tuned to your collector). `[starter]` is a sane default *and* a substitute for the boilerplate during trial / dev — not a substitute for production-grade SDK config.

### Path 1 — Full distributed tracing — `[full]` extra

One install, complete distributed tracing of LLM calls — reliability-layer metrics + span events **and** per-provider HTTP-level child spans with `traceparent` propagated to providers. Auto-instrumentor + `opentelemetry-instrumentation-httpx` activate together.

```bash
pip install 'keel-llm-otel[full]'                # everything wired
opentelemetry-instrument python yourapp.py        # discovers + activates automatically
```

> Why `[full]` is recommended: distributed *tracing* requires HTTP-level instrumentation to inject `traceparent` headers into outgoing provider calls. Installing only `keel-llm-otel[auto]` gives correct reliability-layer metrics + events but **silently** omits per-call child spans and trace propagation — `[full]` eliminates that failure mode.

### Path 1b — Auto-instrument only (reliability layer, no HTTP spans) — `[auto]` extra

For consumers who already use `opentelemetry-instrumentation-httpx` elsewhere, or who want the lighter dep set (no `wrapt` transitively from httpx instrumentation):

```bash
pip install 'keel-llm-otel[auto]'                # adds opentelemetry-instrumentation + wrapt
opentelemetry-instrument python yourapp.py        # discovers + activates automatically
```

**Or programmatically** — one line at startup:

```python
from keel_llm_otel import OTelInstrumentor
OTelInstrumentor().instrument()    # call once; ResilientClient is patched globally
```

Your existing code is unchanged:

```python
from keel_llm_reliability import ResilientClient, Request
client = ResilientClient([...])
result = await client.failover(req)   # metrics + span events emitted automatically
```

### Path 2 — Drop-in subclass (one-word code change) — lean install

`pip install keel-llm-otel` (no extras needed) gets you a `ResilientClient` subclass with emission built in. When you want explicit, per-instance control (multi-tenant meters, test isolation, custom Tracer):

```python
- from keel_llm_reliability import ResilientClient
+ from keel_llm_otel import OTelObservableClient as ResilientClient
```

**Same constructor, same methods, same return types** — `OTelObservableClient` *is* a `ResilientClient`:

```python
import asyncio
from keel_llm_otel import OTelObservableClient as ResilientClient
from keel_llm_adapter_openai import OpenAIAdapter
from keel_llm_reliability import Request
from keel_llm_protocol import user

client = ResilientClient([                 # same call as before
    OpenAIAdapter(model="llama-3.3-70b-versatile", api_key="gsk_…",
                  base_url="https://api.groq.com/openai/v1", provider="groq"),
    OpenAIAdapter(model="llama-3.1-8b", base_url="http://localhost:11434/v1",
                  provider="local"),
])

async def main() -> None:
    result = await client.failover(Request(messages=[user("Hello.")]))
    # ↑ identical behaviour; metrics + span events emitted as a side-effect.

asyncio.run(main())
```

Both paths produce identical signals; both use OTel's no-op meter/tracer when nothing is configured (zero overhead, zero failure).

### Path 3 — Direct emission (decoupled from `keel-llm-reliability`'s orchestrator)

If you have your own multi-model orchestrator (in-tree fan-out / failover code), you can still emit on the same OTel vocabulary — `Signals.emit(...)` accepts any iterable of `AttemptLike` (a structural Protocol: anything with `model_key` / `outcome` / `latency_ms` / `error`). No `keel-llm-reliability` adoption required.

```python
from keel_llm_otel import Signals
# Build your own attempt records (any shape with the four fields above):
attempts = [
    MyAttempt(model_key="gpt-4o", outcome="success", latency_ms=120, error=None),
    MyAttempt(model_key="claude-sonnet", outcome="deferred_backpressure",
              latency_ms=45, error=MyError(category="backpressure")),
]
Signals().emit(strategy="my_orchestrator", attempts=attempts,
               succeeded=True, degraded=True)
```

Useful when adopting Keel's full orchestrator isn't the right move (you have working in-tree machinery), but you still want unified OTel emission across consumers in your portfolio.

## What gets emitted

Configure your OTel SDK (exporters, resource attributes, sampling) however you normally would — this package only *produces* the signals.

### Metrics

| Name | Type | Labels | What it measures |
|---|---|---|---|
| `keel.llm.reliability.attempts` | Counter | `strategy`, `gen_ai.request.model`, `outcome`, `error_category` | One per `Attempt` — count every provider interaction by disposition. |
| `keel.llm.reliability.attempt_duration_ms` | Histogram (ms) | `strategy`, `gen_ai.request.model`, `outcome` | Latency per provider interaction. |
| `keel.llm.reliability.calls` | Counter | `strategy`, `succeeded`, `degraded` | One per reliability call — track end-to-end outcome. |

*(Counter names follow modern OTel convention — no `_total` suffix; the Prometheus exporter adds it automatically. The model-name label uses OTel's incubating GenAI convention `gen_ai.request.model` so reliability-layer dispositions can be joined with adapter-level `gen_ai.*` metrics in cross-layer queries.)*

`strategy` is `"failover"` or `"fan_out"`. `outcome` is one of `success` / `preempted_open` / `preempted_limited` / `deferred_backpressure` / `empty` / `failed`. `error_category` is the failed attempt's `error.category` (`backpressure` / `transient` / `terminal`), or `"none"` when the attempt didn't carry an error.

### Span events

If the caller has a current span (the standard pattern — wrap your reliability call in your own span), each `Attempt` becomes a **`keel.attempt`** event on that span with attributes `keel.strategy`, `gen_ai.request.model`, `keel.outcome`, `keel.latency_ms`, `keel.error_category`:

```python
from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("llm.call") as span:
    result = await client.failover(req)
    # span now carries one `keel.attempt` event per provider interaction —
    # visible in your tracing UI alongside whatever else you put on it.
```

## Design notes

- **Pure wrapper, no coupling.** `keel-llm-reliability` doesn't depend on OTel; this package wraps it externally. Want a different backend (Prometheus, Datadog APM, structured-log emitter)? Write your own ~80-LOC wrapper over `ResilientClient` — the `Attempt` trail is the contract.
- **Post-call emission only.** v0 emits signals after the wrapped call returns, using `latency_ms` from each `Attempt`. No new spans are started. Proper async span-context propagation into provider HTTP calls belongs in adapter-level HTTP instrumentation (e.g. `opentelemetry-instrumentation-httpx`), not in a reliability-layer wrapper.
- **Emission errors are swallowed.** Observability must never break the underlying call. If a meter or tracer raises, the wrapped call's result is still returned.
- **No global-state assumptions.** Pass a specific `Meter` / `Tracer` for tests / multi-tenant; or omit and it picks up the global providers your SDK configured.

## Status

`0.1.0` — `0.x` while the API stabilizes through year one (breaking changes possible at minor bumps, documented in the CHANGELOG; **pin exact versions**).

## The Keel toolkit

Composable, vendor-neutral LLM reliability libraries on PyPI:
[`keel-llm-reliability`](https://pypi.org/project/keel-llm-reliability/) · [`keel-llm-protocol`](https://pypi.org/project/keel-llm-protocol/) · [`keel-llm-adapter-openai`](https://pypi.org/project/keel-llm-adapter-openai/) · [`keel-llm-adapter-anthropic`](https://pypi.org/project/keel-llm-adapter-anthropic/) · [`keel-llm-adapter-google`](https://pypi.org/project/keel-llm-adapter-google/) · [`keel-circuit-breaker`](https://pypi.org/project/keel-circuit-breaker/) · [`keel-llm-otel`](https://pypi.org/project/keel-llm-otel/)

MIT licensed.
