Metadata-Version: 2.4
Name: auscope-sdk
Version: 0.1.0
Summary: Fire-and-forget LLM output auditing — wrap your existing SDK client, get a verified/uncertain/unreliable verdict on every response.
Project-URL: Homepage, https://github.com/auscope/auscope
Project-URL: Documentation, https://github.com/auscope/auscope/blob/main/sdk/README.md
Author: Auscope
License-Expression: MIT
Keywords: anthropic,audit,guardrails,hallucination,llm,observability,openai
Requires-Python: >=3.11
Requires-Dist: httpx>=0.28.1
Requires-Dist: pydantic>=2.0
Provides-Extra: all
Requires-Dist: agent-framework-core>=1.9.0; extra == 'all'
Requires-Dist: anthropic>=0.40.0; extra == 'all'
Requires-Dist: azure-ai-inference>=1.0.0b4; extra == 'all'
Requires-Dist: google-generativeai>=0.8.0; extra == 'all'
Requires-Dist: langchain-core>=0.3.0; extra == 'all'
Requires-Dist: openai>=1.50.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40.0; extra == 'anthropic'
Provides-Extra: azure
Requires-Dist: azure-ai-inference>=1.0.0b4; extra == 'azure'
Provides-Extra: google
Requires-Dist: google-generativeai>=0.8.0; extra == 'google'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain'
Provides-Extra: maf
Requires-Dist: agent-framework-core>=1.9.0; extra == 'maf'
Provides-Extra: openai
Requires-Dist: openai>=1.50.0; extra == 'openai'
Description-Content-Type: text/markdown

# auscope

Fire-and-forget LLM output auditing. Wrap the LLM client you already use — no
call-site rewrites — and every response gets sent to an Auscope audit server
in the background. Get back a `verified` / `uncertain` / `unreliable` verdict,
a confidence score, reasoning, and real per-call cost, without blocking your
response to the user.

```python
import anthropic
from auscope.adapters.anthropic import AuscopeAnthropic

client = AuscopeAnthropic(
    anthropic.AsyncAnthropic(api_key="..."),
    audit_url="https://api.your-auscope-instance.com",
    api_key="asc_live_...",
)

response = await client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)
# response is exactly what anthropic.AsyncAnthropic would have returned —
# the audit fired in the background, doesn't block this call.

verdict = await client.last_verdict()
print(verdict.verdict, verdict.confidence_score, verdict.cost_usd)
```

## Install

```bash
pip install auscope-sdk[anthropic]      # or [openai], [azure], [google], [langchain], [maf], [all]
```

Each extra installs the one provider SDK you need. Core install (`pip install
auscope-sdk`) only pulls `httpx` + `pydantic` — bring your own provider SDK if
you don't want an extra. The package is `auscope-sdk` on PyPI; you still
`import auscope` in code.

## Supported adapters

| Provider | Import | Wraps | Drop-in method |
|---|---|---|---|
| Anthropic | `auscope.adapters.anthropic.AuscopeAnthropic` | `anthropic.AsyncAnthropic` | `client.messages.create(...)` |
| OpenAI | `auscope.adapters.openai.AuscopeOpenAI` | `openai.AsyncOpenAI` | `client.chat.completions.create(...)` |
| OpenRouter | `auscope.adapters.openrouter.AuscopeOpenRouter` | `openai.AsyncOpenAI` (OpenRouter base_url) | `client.chat.completions.create(...)` |
| Azure AI Inference | `auscope.adapters.azure.AuscopeAzure` | `azure.ai.inference.aio.ChatCompletionsClient` | `client.complete(...)` |
| Google Generative AI | `auscope.adapters.google.AuscopeGoogle` | `google.generativeai.GenerativeModel` | `client.generate_content_async(...)` |
| LangChain | `auscope.adapters.langchain.AuscopeLangChain` | any LangChain `BaseChatModel` | `client.ainvoke(...)` |
| MAF (Multi-Agent Framework) | `auscope.adapters.maf.AuscopeMAF` | an `agent_framework.Agent` | `client.run(...)` |
| Any provider | `auscope.adapters.raw.auscope_watch` | a plain `async def fn(prompt) -> str` | `@auscope_watch(...)` decorator |

Every adapter is a thin subclass — anything you'd normally call on the real
SDK client still works via passthrough (`__getattr__`). Streaming (`stream=True`)
is supported on the Anthropic and OpenAI/OpenRouter adapters: chunks pass
through unchanged, the audit fires once with the full accumulated text after
the stream is exhausted.

## Sync clients

If your codebase uses the synchronous SDK variants (`openai.OpenAI`,
`anthropic.Anthropic`) instead of the `Async*` ones:

```python
from openai import OpenAI
from auscope.sync import AuscopeOpenAISync

client = AuscopeOpenAISync(OpenAI(api_key="..."), audit_url="...", api_key="asc_live_...")
response = client.chat.completions.create(model="gpt-4o", messages=[...])
verdict = client.last_verdict()  # blocks until the audit completes
```

Audits still fire in the background on a shared thread — the call returns
immediately. No streaming support in the sync variant yet.

## Constructor options

Every adapter (and `auscope_watch`) accepts:

| Kwarg | Default | Meaning |
|---|---|---|
| `audit_url` | `http://localhost:8000` | Your Auscope server. |
| `api_key` | `None` | Auscope API key (`asc_live_...`), sent as `Authorization: Bearer`. |
| `domain` | `"general"` | Audit domain hint (e.g. `"medical"`, `"legal"`) passed to the council. |
| `allow_search` | `False` | Let auditor models use web search to verify claims. |
| `models` | server default | Override which council models audit this client's calls. |
| `chairman_model` | server default | Override the chairman model. |
| `system_prompt` | `None` | Your bot's system prompt, included as audit context. |
| `chat_history` | `None` | Prior conversation turns, included as audit context. |
| `on_audit_error` | `None` | `Callable[[Exception], None]` — called if a background audit fails, even if you never call `last_verdict()`. |
| `sample_rate` | `1.0` | Fraction of calls to audit (0.0–1.0). Below 1.0, skips auditing at random — useful for high-volume endpoints where auditing every call is too expensive. |
| `max_retries` | `2` | Retries on transient network errors / 5xx from the audit server. |
| `retry_backoff_base` | `0.5` | Seconds, doubles each retry. |

Per-call overrides: pass `auscope_system_prompt=...` / `auscope_chat_history=...`
as extra kwargs to any wrapped call (`create_message`, `create_completion`,
etc.) to override the instance defaults for that one call — useful for a
single client instance serving a multi-tenant bot with different system
prompts per request. These are popped before forwarding to the real SDK, so
they never reach the provider.

## Reading the verdict

`await client.last_verdict()` (or `client.last_verdict()` for sync clients)
returns the oldest not-yet-consumed audit, FIFO per client instance:

```python
verdict = await client.last_verdict()
verdict.verdict            # "verified" | "uncertain" | "unreliable"
verdict.confidence_score   # 0.0-1.0
verdict.reasoning          # chairman's explanation
verdict.cost_usd           # real $ cost of the audit council + chairman calls
verdict.model_calls        # list[ModelResponse] — per-auditor tokens/cost/latency
```

If you don't need the verdict inline, don't call `last_verdict()` — the audit
still runs and fires `on_audit_error` on failure; nothing blocks.

## No SDK for your provider?

Skip the adapter, audit directly:

```python
from auscope.base import AuscopeBase

client = AuscopeBase(audit_url="...", api_key="asc_live_...")
verdict = await client.audit_direct(query="What is the capital of France?", llm_response="Paris.")
```

## Development (this monorepo)

This package lives at `sdk/` in the Auscope monorepo and is a `uv` workspace
member — the root project depends on it via `auscope = { workspace = true }`,
so `import auscope` works locally without publishing. Tests: `sdk/tests/`
(`uv run pytest sdk/tests`). Live provider tests are skipped unless the
matching `*_API_KEY` env var is set (see `sdk/.env.example`).

## Publishing

```bash
cd sdk
uv build
uv publish   # needs PYPI_TOKEN / --token
```
