Metadata-Version: 2.4
Name: qagent-tracer
Version: 0.2.0
Summary: Lightweight tool execution tracing SDK for evaluating AI agents on QAgent.
Author-email: QAgent Team <support@qagent.in>
License-Expression: MIT
Project-URL: Homepage, https://qagent.in
Project-URL: Documentation, https://qagent.in/docs
Project-URL: Repository, https://github.com/qagent-ai/qagent-sdk
Keywords: qagent,evaluation,llm,ai-agents,tool-calling,function-calling,tracing,observability
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# QAgent Tracer

Lightweight tool execution tracing SDK for evaluating AI agents on [QAgent](https://qagent.in).

`qagent-tracer` captures, timestamps, isolates, and redacts tool executions within your AI agent backend so the QAgent platform can rigorously evaluate tool calling accuracy, parameter fidelity, hallucination, and task completion.

---

## Features

- Zero third-party dependencies (built entirely on Python standard library: `contextvars`, `functools`, `inspect`, `re`, `json`, `time`).
- Thread-safe and asynchronous-safe request isolation via `contextvars`.
- Automatic PII and credential redaction (API keys, Bearer tokens, passwords, credit cards, emails, phone numbers).
- Safe payload truncation for oversized tool returns (prevents memory spikes without breaking agent code).
- Resilient serialization (handles non-JSON serializable objects like `datetime` or database models safely without crashing).
- Negligible runtime overhead (< 0.1ms per call).

---

## Installation

```bash
pip install qagent-tracer
```

Requirements: Python 3.8 or higher.

---

## Quickstart

### 1. Decorate Your Tools

Wrap any tool or helper function with `@trace_tool`:

```python
from qagent_tracer import trace_tool
# (or: from qagent_sdk import trace_tool)

@trace_tool
def check_account_status(user_email: str) -> dict:
    # Your database query or external API call
    return {
        "plan": "Solo",
        "monthly_cost": 15,
        "runs_used": 84,
        "runs_total": 200,
        "active": True
    }
```

Async functions are fully supported:

```python
@trace_tool
async def fetch_user_orders(user_id: str) -> list:
    # Asynchronous external call
    return [{"order_id": "ORD-101", "status": "shipped"}]
```

### 2. Return Traces in Your Webhook Response

At the start of your request handler, initialize clean storage with `clear_traces()`. After your agent generates its answer, retrieve the captured traces with `get_traces()` and include them under `"traces"` in your JSON response:

```python
from fastapi import FastAPI
from pydantic import BaseModel
from typing import List, Dict, Any, Optional
from qagent_sdk import clear_traces, get_traces

app = FastAPI()

class ChatRequest(BaseModel):
    messages: List[Dict[str, str]]
    user_id: Optional[str] = None

@app.post("/chat")
async def chat(request: ChatRequest):
    # 1. Reset trace context for this incoming request
    clear_traces()

    # 2. Run your agent logic (LLM invocation + tool execution)
    user_message = request.messages[-1]["content"]
    agent_reply = await run_my_agent(user_message)

    # 3. Retrieve captured tool traces (automatically redacted by default)
    traces = get_traces()

    # 4. Return reply alongside traces
    return {
        "reply": agent_reply,
        "traces": traces
    }
```

---

## Complete FastAPI Example

Here is a complete, runnable agent webhook:

```python
from fastapi import FastAPI
from pydantic import BaseModel
from typing import List, Dict, Any
from qagent_sdk import trace_tool, clear_traces, get_traces, set_current_tool_call_id

app = FastAPI(title="Customer Support Agent")

# Define tools
@trace_tool
def check_account_status(user_email: str) -> dict:
    return {
        "email": user_email,
        "plan": "Solo",
        "runs_used": 84,
        "runs_total": 200,
        "active": True
    }

@trace_tool
def escalate_to_human(user_email: str, reason: str, priority: str = "normal") -> dict:
    return {
        "ticket_id": "TICK-9081",
        "status": "queued",
        "priority": priority
    }

class Message(BaseModel):
    role: str
    content: str

class ChatPayload(BaseModel):
    messages: List[Message]

@app.post("/chat")
async def chat_handler(payload: ChatPayload):
    clear_traces()

    last_user_message = payload.messages[-1].content

    # Example: When the LLM requests a tool call with an ID
    tool_call_id = "call_abc123"
    set_current_tool_call_id(tool_call_id)

    # Execute the tool
    result = check_account_status(user_email="alex@startup.io")

    # Generate response
    reply = (
        f"Your account is on the {result['plan']} plan. "
        f"You have used {result['runs_used']} of {result['runs_total']} test runs."
    )

    return {
        "reply": reply,
        "traces": get_traces()
    }
```

---

## Flask Integration

```python
from flask import Flask, request, jsonify
from qagent_sdk import trace_tool, clear_traces, get_traces

app = Flask(__name__)

@trace_tool
def query_knowledge_base(query: str) -> dict:
    return {"results": ["Sample doc content"]}

@app.route("/chat", methods=["POST"])
def chat():
    clear_traces()
    data = request.get_json()

    # Run agent execution
    reply = "Found relevant docs."
    query_knowledge_base(data["messages"][-1]["content"])

    return jsonify({
        "reply": reply,
        "traces": get_traces()
    })
```

---

## Tool Call ID Correlation

When using OpenAI, Anthropic, or LangChain agents, the model assigns a unique `tool_call_id` (e.g., `"call_98x...""`). To link the physical tool execution to the model's intent:

```python
from qagent_sdk import set_current_tool_call_id, trace_tool

# Set before calling the tool function
set_current_tool_call_id(tool_call.id)
result = my_tool(**tool_call.arguments)
```

The resulting trace will record:
- `tool_call_id`: The matching ID string.
- `correlation_confidence`: `"HIGH"`.

If no ID is set, the SDK automatically generates a local identifier with `"LOW"` correlation confidence.

---

## Security & Sensitive Data Redaction

By default, `get_traces(redact=True)` automatically sanitizes sensitive fields before returning them:

- API Keys (`sk-...`, `gsk_...`, Anthropic, AWS tokens)
- Bearer Authorization Headers
- Passwords, secret tokens, and credentials
- Credit card numbers (Luhn 13-19 digit patterns)
- Email addresses and phone numbers

Identifiers like `order_id`, `user_id`, `ticket_id`, and `customer_id` remain intact so evaluation rubrics can verify parameter values.

To retrieve unredacted traces internally (e.g., for custom debugging):

```python
from qagent_sdk import get_raw_traces

raw = get_raw_traces()
```

---

## Privacy & Masking

`qagent-tracer` supports three privacy and masking levels:

### 1. Default Mode (Strict Redaction)
- **Behavior**: All secrets, credentials, API keys, credit cards, emails, and phone numbers are completely masked (e.g. `[REDACTED:email]`, `[REDACTED:phone]`, `[REDACTED:api_key]`).
- **Configuration**: Default behavior. Zero configuration required.

### 2. Fingerprint Mode (Opt-in Verification)
- **Behavior**: Secrets and payment cards remain strictly masked (`[REDACTED:api_key]`, `[REDACTED:card_number]`, etc.). Identifiers (emails and phone numbers) are masked with a deterministic HMAC-SHA256 fingerprint:
  - `[REDACTED:email:fp=<12 hex chars>]`
  - `[REDACTED:phone:fp=<12 hex chars>]`
- **Use Case**: Allows the QAgent evaluation engine to verify whether the agent used the *expected* customer email or phone number across multi-turn interactions without ever exposing the real PII value.
- **Configuration**:
  - Environment variable: `QAGENT_FINGERPRINT_KEY="your-secret-key"`
  - Code:
    ```python
    import qagent_sdk

    qagent_sdk.configure(fingerprint_key="your-secret-key")
    ```

### 3. Test Mode (Opt-in Staging)
- **Behavior**: For mock/staging test accounts only. Emails and phone numbers are left unmasked so developers can inspect raw values during testing. Critical secrets (API keys, passwords, bearer tokens, credit cards) **remain strictly masked**. Logs a one-time safety warning:
  `"qagent-tracer test mode: emails and phone numbers are not masked. Do not use with real customer data."`
- **Configuration**:
  - Environment variable: `QAGENT_TEST_MODE=true`
  - Code:
    ```python
    import qagent_sdk

    qagent_sdk.configure(test_mode=True)
    ```

---

## Trace JSON Schema

The traces emitted by `get_traces()` adhere to this structure:

```json
[
  {
    "tool_call_id": "call_abc123",
    "tool_name": "check_account_status",
    "input": {
      "user_email": "alex@startup.io"
    },
    "output": {
      "plan": "Solo",
      "runs_used": 84,
      "runs_total": 200,
      "active": true
    },
    "status": "success",
    "error": null,
    "duration_ms": 14.2,
    "timestamp": 1727725482.102,
    "correlation_confidence": "HIGH"
  }
]
```

---

## Changelog

### 0.2.0 (2026-10-08)
- Added opt-in HMAC-SHA256 fingerprinting for emails and phone numbers via `QAGENT_FINGERPRINT_KEY` and `configure(fingerprint_key="...")`.
- Added opt-in test mode via `QAGENT_TEST_MODE=true` and `configure(test_mode=True)` with one-time safety warning logging.
- Added `configure()` exported from both `qagent_sdk` and `qagent_tracer`.
- Full backward compatibility: 0.1.0 behavior remains default when no new settings are provided.
- Preserved strict non-negotiable masking for all API keys, bearer tokens, passwords, secrets, credentials, CVVs, and credit cards across all modes.

---

## License

MIT License. See [LICENSE](LICENSE) for details.
