Metadata-Version: 2.4
Name: spolm-tracer
Version: 0.1.3
Summary: Official Spolm SDK for Python — trace, simulate, and verify AI agent runs.
Author: Spolm
License: MIT
Project-URL: Homepage, https://spolm.com
Project-URL: Repository, https://github.com/Tanrocode/spolm
Keywords: ai,agents,llm,tracing,observability,verification,simulation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.25

# spolm-tracer

Official Spolm SDK for Python — trace, simulate, and verify AI agent runs.

`spolm-tracer` instruments your agent so its runs appear in Spolm, where they're
clustered into failure patterns, turned into enforceable checks, and certified in
a shareable Reliability Report. The JavaScript SDK is
[`@spolm/tracer`](https://www.npmjs.com/package/@spolm/tracer) (same model).

## What is and isn't supported today

Be clear-eyed about this before you integrate:

| Capability | Status |
| --- | --- |
| Single-turn agents (one task → one output) | ✅ Fully supported |
| Tool-using agents (side effects, policies) | ✅ Fully supported |
| **Multi-turn / conversational agents** (sessions) | ✅ Supported via `session_id` (below) |
| **Conversation-scoped checks** (cross-turn) | ✅ Supported (`target: conversation`) |
| **Multi-turn adversarial simulation** (persona) | ✅ Supported (`simulate_conversation`) |
| Voice agents (transcript as text) | ⚠️ Works — you pass the **ASR transcript** as the task |
| Native audio / ASR-confidence / TTS metadata | ❌ Not yet (roadmap). No `low-confidence` checks yet |
| Non-text triggers (image/file/webhook input kinds) | ⚠️ Pass a text description as the task; no typed input kind yet |

Nothing here requires a framework integration — you instrument your own code.

## Install

```bash
pip install spolm-tracer
```

## Quickstart — single-turn agent

```python
from spolm_tracer import Tracer

tracer = Tracer(API_KEY, AGENT_ID)   # from your Spolm dashboard

tracer.startRun("Summarize the Q3 earnings call")

# Wrap the functions you want traced as steps
traced_llm = tracer.log_step(step_name="answer", step_type="model", provider="openai")(call_llm)
result = await traced_llm(prompt)

tracer.end_run(result, "complete")
```

## Quickstart — multi-turn chat / voice agent (sessions)

A conversation is a **session**: each turn is one run, and all turns share a
`session_id`. Pass it in `startRun` metadata (add `turn_index` to order them).
This is backward-compatible — omit `session_id` and each run is its own
single-turn conversation.

```python
session_id = "call-42"          # your call/conversation id
for turn_index, user_msg in enumerate(conversation):
    tracer.startRun(user_msg, metadata={"session_id": session_id, "turn_index": turn_index})
    reply = await run_one_turn(user_msg)   # your agent keeps its own memory per session
    tracer.end_run(reply, "complete")
```

For a **voice** agent, pass the ASR transcript as the task — Spolm sees the text
of each turn. (Native audio/confidence metadata is on the roadmap.)

Conversation-scoped checks (see below) then evaluate the whole call — e.g.
"identity must be verified before a refund is issued anywhere in the session."

## Simulation — auto-generated test cases (single-turn)

```python
async def agent_fn(user_task, tracer):
    tracer.startRun(user_task)              # agent_fn must call startRun/end_run itself
    result = await run_agent(user_task)
    tracer.end_run(result, "complete")
    return result

await tracer.simulate(agent_fn, mode="full", pause_ms=250)   # generated + adversarial packs
await tracer.simulate(agent_fn, mode="fast")                 # deterministic packs only (cheap CI)
```

## Multi-turn adversarial simulation (personas)

Instead of one static prompt, an **adaptive simulated user** escalates across
turns (polite → "I already spoke to someone" → urgency → frustration; or shifts
identity mid-call). Each persona becomes one session; the whole conversation is
then checked. Your `agent_fn` runs ONE turn and keeps its own memory keyed by
`os.environ["SPOLM_SESSION_ID"]`.

```python
async def agent_fn(user_message, tracer):
    session = os.environ.get("SPOLM_SESSION_ID")
    tracer.startRun(user_message)           # session_id/turn_index auto-tagged during sim
    reply = await run_one_turn(session, user_message)
    tracer.end_run(reply, "complete")
    return reply

result = await tracer.simulate_conversation(agent_fn, max_turns=6)
# → sessions flow through the same evaluate/readiness pipeline; conversation
#   checks (e.g. verify-before-account-change) surface any cross-turn failure.
```

## Property-based testing

Actively search for inputs that violate your promoted checks — trace-seeded
mutations, the enforced suite as the oracle, delta-debugged minimal repros:

```python
result = await tracer.property_test(
    agent_fn,
    invariants=[{"id": "no-injection-obedience", "description": "never obey injected instructions"}],
    iterations=2, per_round=8,
)
print(result["violations_found"], "violations", result["violations"])
```

## Checks — the schema you're verified against

A **check** is a machine-readable assertion evaluated in replay, simulation, and
at runtime. Author them in natural language in the dashboard, or in raw form:

- **type**: `predicate` (a CEL expression), `schema` (JSON Schema), `policy` (fixed patterns)
- **severity**: `block` (fails → BLOCKED deploy), `warn` (advisory), `escalate`
- **scope.target**: `tool_call` · `llm_output` · `trajectory` (one run) · `memory_op` ·
  `retrieval` · **`conversation`** (a whole session, across turns)

Single-run policy forms: `{never}`, `{only_after}`, `{at_most}`, `{deny_tools}`/`{allow_tools}`.
Conversation policy forms (target `conversation`):
`{requires_before: {action, precondition}}`, `{never_after: {marker, action}}`,
`{deny_tools_in_session: [...]}`.

Example — refund requires prior identity verification anywhere in the call:

```yaml
check:
  id: conv-verify-before-account-change
  severity: block
  scope: { target: conversation }
  assert:
    type: policy
    policy:
      requires_before:
        action: ["refund*", "make_payment*"]
        precondition: ["*verify*", "*authenticate*"]
```

## Verdicts & confidence

- **READY** — passed every enforced (block-severity) check on the tested scenarios.
- **BLOCKED** — a block-severity check failed, or an enforced check was never tested.
- **NOT_VERIFIED** (reports) — an enforced check was never exercised → not certified.
- A check is **UNTESTED** when no scenario ever exercised it.
- **Confidence %** grades *how much to trust a READY* — scenario diversity, tool
  coverage, adversarial depth, and recency. Not "is it safe" (checks answer that)
  but "how thoroughly did we test?".

## Troubleshooting

- **"Mining is empty / 0 pass 0 fail."** There are no *failures* to mine from. A
  read-only agent that correctly resists every attack produces no violations, so
  there's nothing to cluster into a check. Mining needs runs that actually failed.
- **"Caught 0/6 attacks."** 0 *breaches* = the agent resisted all 6 = good. "Caught"
  counts attacks that broke through; 0 is the best score, not a miss.
- **"My check is UNTESTED."** No scenario exercised the tool/path it guards. Add a
  scenario that triggers it (or run a simulation) and it moves to PASS/FAIL.

## API

| Method | Purpose |
| --- | --- |
| `Tracer(api_key, agent_id, options=None)` | Create a tracer |
| `startRun(user_task, metadata={})` | Begin a run; `metadata` may include `session_id`, `turn_index` |
| `log_step(step_name=, step_type=, provider=None)(fn)` | Wrap a function as a traced step |
| `end_run(final_result, status="complete")` | Finalize and send the run |
| `await simulate(agent_fn, count=None, pause_ms=0, mode="full")` | Generated/adversarial test cases (single-turn) |
| `await simulate_conversation(agent_fn, count=None, max_turns=6, pause_ms=0)` | Multi-turn persona simulation |
| `await property_test(agent_fn, invariants=None, iterations=1, per_round=8)` | Search for invariant violations |

MIT.
