Metadata-Version: 2.4
Name: adeptiv-ai-evaluator-sdk
Version: 1.58.0.dev1
Summary: Adeptiv GenAI tracing SDK for LLMs, agents, and AI workflows
Author-email: Adeptiv AI <contact@adeptiv-ai.com>
License: Apache-2.0
Description-Content-Type: text/markdown
Requires-Dist: opentelemetry-api
Requires-Dist: opentelemetry-sdk
Requires-Dist: opentelemetry-exporter-otlp-proto-common
Requires-Dist: opentelemetry-instrumentation-logging
Requires-Dist: websockets<17,>=13
Provides-Extra: gateway
Requires-Dist: httpx>=0.27; extra == "gateway"
Provides-Extra: openai
Requires-Dist: openinference-instrumentation-openai; extra == "openai"
Provides-Extra: openai-agents
Requires-Dist: openinference-instrumentation-openai-agents; extra == "openai-agents"
Provides-Extra: anthropic
Requires-Dist: openinference-instrumentation-anthropic; extra == "anthropic"
Provides-Extra: litellm
Requires-Dist: openinference-instrumentation-litellm; extra == "litellm"
Provides-Extra: langchain
Requires-Dist: openinference-instrumentation-langchain; extra == "langchain"
Provides-Extra: llama-index
Requires-Dist: openinference-instrumentation-llama-index; extra == "llama-index"
Provides-Extra: vertexai
Requires-Dist: openinference-instrumentation-vertexai; extra == "vertexai"
Provides-Extra: bedrock
Requires-Dist: openinference-instrumentation-bedrock; extra == "bedrock"
Provides-Extra: google-genai
Requires-Dist: openinference-instrumentation-google-genai; extra == "google-genai"
Provides-Extra: mistralai
Requires-Dist: openinference-instrumentation-mistralai; extra == "mistralai"
Provides-Extra: groq
Requires-Dist: openinference-instrumentation-groq; extra == "groq"
Provides-Extra: dspy
Requires-Dist: openinference-instrumentation-dspy; extra == "dspy"
Provides-Extra: crewai
Requires-Dist: openinference-instrumentation-crewai; extra == "crewai"
Provides-Extra: haystack
Requires-Dist: openinference-instrumentation-haystack; extra == "haystack"
Provides-Extra: smolagents
Requires-Dist: openinference-instrumentation-smolagents; extra == "smolagents"
Provides-Extra: autogen
Requires-Dist: openinference-instrumentation-autogen; extra == "autogen"
Provides-Extra: instructor
Requires-Dist: openinference-instrumentation-instructor; extra == "instructor"
Provides-Extra: all
Requires-Dist: openinference-instrumentation-openai; extra == "all"
Requires-Dist: openinference-instrumentation-openai-agents; extra == "all"
Requires-Dist: openinference-instrumentation-anthropic; extra == "all"
Requires-Dist: openinference-instrumentation-litellm; extra == "all"
Requires-Dist: openinference-instrumentation-langchain; extra == "all"
Requires-Dist: openinference-instrumentation-llama-index; extra == "all"
Requires-Dist: openinference-instrumentation-vertexai; extra == "all"
Requires-Dist: openinference-instrumentation-bedrock; extra == "all"
Requires-Dist: openinference-instrumentation-google-genai; extra == "all"
Requires-Dist: openinference-instrumentation-mistralai; extra == "all"
Requires-Dist: openinference-instrumentation-groq; extra == "all"
Requires-Dist: openinference-instrumentation-dspy; extra == "all"
Requires-Dist: openinference-instrumentation-crewai; extra == "all"
Requires-Dist: openinference-instrumentation-haystack; extra == "all"
Requires-Dist: openinference-instrumentation-smolagents; extra == "all"
Requires-Dist: openinference-instrumentation-autogen; extra == "all"
Requires-Dist: openinference-instrumentation-instructor; extra == "all"

# Adeptiv GenAI Tracing SDK

**Adeptiv GenAI** is a lightweight, vendor-neutral **OpenTelemetry tracing SDK** for **LLMs, agents, and AI workflows**.

It captures **latency, execution flow, hashes, and metadata** for all AI use cases.

---

## Key Features

-  **Trace-only observability** (no response normalization)
- Works with **any LLM** (OpenAI, Anthropic, Gemini, local models)
- Supports **chat, summarization, agents, tools, RAG**
- **Sync & async** compatible
- **Enterprise-safe** (hashing, redaction, previews)
- **OpenTelemetry compatible**
---

This ensures **zero lock-in** and **maximum compatibility**.

---

## Installation

```bash

pip install adeptiv-ai-evaluator-sdk

```

# Adeptiv GenAI Tracing SDK


Tracing an LLM Call

```python


SDK configuration

config.endpoint = "wss://<adeptiv host>/ws/sdk/" # Adeptiv endpoint (set before the key)
config.api_key = "" #project Key 
config.project_name = "customer-support-bot" # Project Name


from adeptiv_evaluator_sdk import trace_llm

@trace_llm(model="gpt-4o", operation="chat", workflow_id="")
def chat(prompt):
    return llm.invoke(prompt)

chat("Hello world")

```


```python
# Adeptiv GenAI Tracing Agents SDK

from adeptiv_evaluator_sdk import trace_llm

@trace_agent("support_agent", model="gpt-4o",workflow_id="")
async def run_agent(query):
    return await agent.run(query)


```


---

## Containment

When a containment rule in Adeptiv acts on an agent (pause it, block a tool,
allow only approved models, slow it down), the SDK enforces it inside your
process. Only the agent the rule acted on is affected; every other agent keeps
working.

The SDK keeps the use case's policy in memory; Adeptiv pushes it over the
SDK's WebSocket when it connects and again whenever it changes, so a check
costs about a microsecond and never makes a network call. An action stays until someone resumes the agent in
Adeptiv; if Adeptiv cannot be reached, the last policy stays in force. If the
policy has never been fetched, nothing is enforced: an Adeptiv outage never
stops your agents.

Every `@trace` call is checked (`contained=True` is the default); pass
`contained=False` for a call that must never be stopped. Name agents the way
they are named in Adeptiv:

```python
from adeptiv_evaluator_sdk import trace, ContainmentError

@trace(operation="agent", agent_name="Refund Agent")
def refund_agent(question):             # pause, approved models, slow down
    ...

@trace(operation="tool")
def issue_refund(order_id):             # blocked for the current agent?
    ...

@trace(operation="chat", model="gpt-4.1")
def ask_model(prompt):                  # outside the approved models?
    ...

try:
    reply = refund_agent(question)
except ContainmentError as exc:        # AgentPaused, ToolBlocked, ModelNotApproved, RateLimited
    reply = exc.fallback_message       # "A team member will reply to you shortly." or the rule's message
```

A refused call still shows on its span, marked with the action. A contained
agent call makes its agent the current one, so contained tool and model calls
inside it are checked for that agent.

For a tool dispatcher or a model call you cannot trace, check it before it
runs; the agent is the one of the enclosing contained agent call:

```python
from adeptiv_evaluator_sdk import guard_tool, guard_model

for call in response.tool_calls:
    guard_tool(call.name)              # raises ToolBlocked if this tool is blocked
    run_tool(call)
guard_model("gpt-4.1")                 # raises ModelNotApproved outside the approved list
```

| Setting | Env var | Default |
|---|---|---|
| `containment_enabled` | `ADEPTIV_CONTAINMENT` | `true` |
| `containment_initial_wait_seconds` | `ADEPTIV_CONTAINMENT_INITIAL_WAIT` | `1` (first call of a process only) |
| `containment_max_wait_seconds` | `ADEPTIV_CONTAINMENT_MAX_WAIT` | `10` (slow-down wait before `RateLimited`) |

Inside a `@trace` span, a refused or delayed call marks it with `adeptiv.containment.action`,
`adeptiv.containment.outcome` (`blocked` / `delayed`) and
`adeptiv.containment.event_id`, so Adeptiv can show the containment held.

---

### LangChain and LangGraph

Nothing to add in a LangGraph app: once the SDK is initialised, every graph
node, tool and model call is checked before it runs. The node name is the
agent, so a pause on "Recommendation Agent" stops the node
`recommendation_agent` and leaves the other nodes running. A refused call
raises out of `graph.invoke`, so one `@trace` around the chatbot is enough:

```python
@trace(operation="agent", name="StoreChatbot")
def chatbot(text):
    try:
        return graph.invoke({"messages": [text]})
    except ContainmentError as exc:
        return exc.fallback_message
```

"Slow it down" is not applied to graph nodes.

## Approval check

With nothing to set up beyond the API key, every model call is checked:
is its model approved in the company's inventory, and is its agent? This
runs on every `@trace` call and on every model call an instrumented
framework makes (LangChain/LangGraph, CrewAI, LlamaIndex, OpenAI Agents,
Google ADK, LiteLLM...). What is not approved is reported to Adeptiv, which
emails the test addresses. It only reports: no call is ever stopped or
delayed, and it takes about 30 microseconds.

```python
import adeptiv_evaluator_sdk as adeptiv
adeptiv.config.api_key = "..."

@adeptiv.trace(operation="agent", agent_name="Support Bot")
def support_bot(question, model):
    return litellm.completion(model=model, messages=[...])
```

- A model is matched to the inventory by its exact name (case aside).
- An agent is matched by the words of its name ("support_bot" is
  "Support Bot"). A `@trace` agent missing from the inventory is reported;
  inside a framework, a call is judged by the inventory agent among the
  names it runs under, and framework names are never reported as agents.
- Turn it off with `ADEPTIV_APPROVAL_CHECK=false`.

## Gateway (optional)

For apps that do not use the SDK in their code, the gateway is a local
server, installed with the SDK, that every model call
of an app is sent to. It works with any agentic app, in any framework or
language, with or without LiteLLM: it reads each provider's own API, checks
the call, forwards it unchanged with the app's own key, and traces it. Apps
that do not use the SDK in their code (another language, n8n, a framework
the SDK does not hook) are traced too.

```bash
pip install "adeptiv-ai-evaluator-sdk[gateway]"
export ADEPTIV_API_KEY=...  ADEPTIV_ENDPOINT=wss://...
adeptiv-gateway --port 4000            # or, inside the app:
```

```python
import adeptiv_evaluator_sdk as adeptiv
adeptiv.config.api_key = "..."
gateway = adeptiv.gateway.start()      # http://127.0.0.1:4000
```

Point the app's model client at it; nothing else changes:

| Provider | Setting |
|---|---|
| OpenAI and OpenAI-compatible hosts | `base_url="http://127.0.0.1:4000/v1"` |
| Azure OpenAI | `azure_endpoint="http://127.0.0.1:4000"` |
| Anthropic (Claude) | `base_url="http://127.0.0.1:4000"` |
| Gemini (google-genai, Google ADK) | `http_options={"base_url": ...}` or `GOOGLE_GEMINI_BASE_URL` |
| LiteLLM | `api_base=` the same URL as the provider above |

Tested with LangGraph (OpenAI and Claude), CrewAI, Agno, the OpenAI Agents
SDK and Google ADK running real tool loops, and with the openai, Azure,
anthropic, google-genai and litellm clients, whole and streamed.

**Where calls go.** Each provider's default URL, or the one in
`ADEPTIV_GATEWAY_OPENAI_UPSTREAM`, `ADEPTIV_GATEWAY_AZURE_UPSTREAM` (or
`AZURE_OPENAI_ENDPOINT`), `ADEPTIV_GATEWAY_ANTHROPIC_UPSTREAM`,
`ADEPTIV_GATEWAY_GEMINI_UPSTREAM`; or per call, the `x-adeptiv-upstream`
header (Groq, OpenRouter, Ollama, vLLM...). A path the gateway does not know
is forwarded as it is: a new provider endpoint never breaks the app.

**Which agent.** Send `x-adeptiv-agent` (and `x-adeptiv-session`,
`x-adeptiv-user`), or put the agent in the URL for a framework that cannot
add headers: `http://127.0.0.1:4000/agent/Refund%20Agent/v1`. One-agent
apps can set `ADEPTIV_GATEWAY_AGENT`. These are never forwarded.

**What it checks.** Only what Adeptiv sends:
- the approval check above, on each call it forwards (the agent named in
  the header or URL);
- with `ADEPTIV_GATEWAY_CONTAINMENT=true` (off until gateway rules are
  configured), the containment actions in force (pause, approved models,
  slow down) refuse a call before it is sent, with an error in the
  provider's own format; a model's answer asking for a blocked tool is
  withheld (in a stream already sent, it is reported).

The checks take about 30 microseconds per call, with 5,000 rules; the
gateway as a whole adds about 4 ms (the local hop).

## Transport

Traces and the containment policy share **one WebSocket per process** to
Adeptiv, opened in the background. You give the SDK two things: the endpoint
and your project API key.

```python
from adeptiv_evaluator_sdk import init_telemetry

init_telemetry(
    api_key="<project api key>",
    endpoint="wss://<adeptiv host>/ws/sdk/",
)
```

or `ADEPTIV_API_KEY` and `ADEPTIV_ENDPOINT` in the environment. The key is
sent once, on the `Authorization` header when the connection opens.

Containment changes are pushed as they happen. While the connection is down
your app never waits on it: trace batches are held in memory (up to 32 MB,
gzipped) and sent when it is back, and the last containment policy stays in
force.

| Setting | Env var | Default |
|---|---|---|
| `endpoint` | `ADEPTIV_ENDPOINT` | none (required) |
| `api_key` | `ADEPTIV_API_KEY` | none (required) |
| `realtime_ack_timeout_seconds` | `ADEPTIV_REALTIME_ACK_TIMEOUT` | `10` (then the batch is held and sent again) |
| `realtime_connect_timeout_seconds` | `ADEPTIV_REALTIME_CONNECT_TIMEOUT` | `5` |
| `realtime_max_backoff_seconds` | `ADEPTIV_REALTIME_MAX_BACKOFF` | `60` (reconnect backoff cap) |
