Metadata-Version: 2.4
Name: agent-status-sdk
Version: 1.1.1
Summary: Agent Status SDK - Outside-in monitoring for AI agents
Project-URL: Homepage, https://agentstatus.dev
Project-URL: Dashboard, https://platform.agentstatus.dev
Project-URL: Repository, https://github.com/Carmel-Labs-Inc/agent-status-sdk
Author-email: Carmel Labs <dev@carmel.so>
License-Expression: MIT
License-File: LICENSE
Keywords: agent-status,agents,ai,llm,monitoring,validation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.8
Requires-Dist: httpx>=0.24.0
Requires-Dist: pydantic>=2.0.0
Provides-Extra: all
Requires-Dist: anthropic>=0.18.0; extra == 'all'
Requires-Dist: crewai>=0.1.0; extra == 'all'
Requires-Dist: langchain-core>=0.1.0; extra == 'all'
Requires-Dist: mcp>=0.1.0; extra == 'all'
Requires-Dist: openai>=1.0.0; extra == 'all'
Requires-Dist: pyautogen>=0.2.0; extra == 'all'
Requires-Dist: websockets>=11.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == 'anthropic'
Provides-Extra: autogen
Requires-Dist: pyautogen>=0.2.0; extra == 'autogen'
Requires-Dist: websockets>=11.0; extra == 'autogen'
Provides-Extra: crewai
Requires-Dist: crewai>=0.1.0; extra == 'crewai'
Requires-Dist: websockets>=11.0; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: langchain-core>=0.1.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: websockets>=11.0; extra == 'dev'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.1.0; extra == 'langchain'
Requires-Dist: websockets>=11.0; extra == 'langchain'
Provides-Extra: mcp
Requires-Dist: mcp>=0.1.0; extra == 'mcp'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Provides-Extra: tunnel
Requires-Dist: websockets>=11.0; extra == 'tunnel'
Description-Content-Type: text/markdown

# Agent Status SDK

Outside-in monitoring for AI agents. Residential nodes probe your agent from the real internet — not from a cloud datacenter IP.

Two reach modes:

| Mode | When to use | What you give us |
|------|-------------|------------------|
| **Public URL** | Agent already has a public HTTPS endpoint | The URL |
| **Private (tunnel)** | Agent only lives in a VPC / laptop / closed network | A short-lived connector process next to the agent |

---

## Installation

```bash
pip install agent-status-sdk

# Private / VPC agents (connector deps)
pip install "agent-status-sdk[tunnel]"

# LangChain helpers
pip install "agent-status-sdk[langchain]"

# Everything
pip install "agent-status-sdk[all]"
```

---

## Private agents (tunnel) — the clear path

Use this when nodes **cannot** reach your agent directly.

```
Residential nodes  →  https://rora-tunnel.carmel.so/probe/{agent_id}
                              ↓
                         tunnel relay
                              ↓  (WebSocket to your connector)
                    your process (agent-status expose)
                              ↓
                    http://127.0.0.1:8080  (or any private URL)
```

Nothing inbound to your VPC is required. Your connector dials **out**.

### 1. Create a private agent in the portal

In the Agent Status partner portal: **Add agent → Private (tunnel)**.

You get, **once**:

- `agent_id` (UUID)
- `rtun_…` tunnel token (store it; rotate later if lost)
- a ready-to-run connect command

Monitoring stays paused until a connector attaches.

### 2. Install the connector next to the agent

```bash
pip install "agent-status-sdk[tunnel]"
```

### 3. Run the connector (keep it running)

```bash
agent-status expose \
  --agent-id <agent_uuid> \
  --token rtun_xxx \
  --target http://127.0.0.1:8080
```

- `--target` = the private HTTP base URL only your network can reach
- When connected, residential nodes probe `https://rora-tunnel.carmel.so/probe/<agent_uuid>`
- The relay forwards those requests over the WebSocket into this process, which proxies to `--target`

Same thing from Python:

```python
from agent_status.tunnel import expose_http

expose_http(
    agent_id="<agent_uuid>",
    token="rtun_xxx",
    target="http://127.0.0.1:8080",
)
```

Env alternative for the token: `RORA_TUNNEL_TOKEN`.

### Naming — do not mix these up

| API | What it is |
|-----|------------|
| **`agent-status expose`** / **`expose_http(...)`** | Connector for a **portal-created private agent** (`agent_id` + `rtun_` token). This is the production path. |
| **`from agent_status.integrations.langchain import expose`** | LangChain helper that uses the **same** tunnel under the hood. Prefer `expose(chain, agent_id=..., token="rtun_...")` from the portal — not a random API key as the tunnel token. |

---

## Quick Start (public URL agents)

```python
import agent_status

agent_status.init(api_key="rora_xxx")

agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Support Bot",
    interval_minutes=60,
)

print(f"Registered: {agent.id}")

status = agent_status.status(agent.id)
print(f"Verdict: {status.verdict}")  # UP, DEGRADED, DOWN
print(f"Uptime: {status.uptime_24h}%")
print(f"Latency: {status.latency_p95}ms")
```

## One-Off Validation

```python
result = agent_status.run(
    endpoint="https://api.example.com/chat",
    prompts=["What is 2+2?", "Hello!"],
)

print(f"Verdict: {result.verdict}")
print(f"P95 Latency: {result.latency_p95}ms")
print(f"Pass Rate: {result.pass_rate}")
```

## Authentication (public endpoints)

For public agents that require auth headers on each probe:

```python
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Secured Bot",
    auth={"type": "bearer", "token": "sk-xxx"},
)

agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="API Bot",
    auth={"type": "api_key", "header": "X-API-Key", "value": "xxx"},
)
```

This is **not** the private tunnel. Tunnel auth is the `rtun_` connector token from the portal.

## Advanced Options

```python
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Enterprise Bot",
    interval_minutes=60,
    max_nodes_per_run=10,
    geos=["us", "eu", "ap"],
    timeout_ms=30000,
    eval_type="llm_judge",
    gold_prompt_profile="search_agent",
    inject_geo_context=True,
    streaming=True,
)
```

## LangChain Integration

### Preferred: portal private agent + LangChain chain

```python
from agent_status.integrations.langchain import expose

# Create Private (tunnel) agent in the portal first → copy agent_id + rtun_ token
expose(
    chain,  # your LangChain runnable
    agent_id="<agent_uuid>",
    token="rtun_xxx",
    agent_name="My Support Bot",
)
```

Same tunnel as `agent-status expose`; the connector invokes your chain instead of proxying HTTP.

### Non-blocking

```python
url = expose(
    chain,
    agent_id="<agent_uuid>",
    token="rtun_xxx",
    agent_name="Background Bot",
    blocking=False,
)
print(f"Probe URL: {url}")
```

### Local callback handler (no tunnel)

```python
from langchain_openai import ChatOpenAI
from agent_status.integrations.langchain import AgentStatusCallbackHandler

handler = AgentStatusCallbackHandler(
    api_key="rora_xxx",
    agent_name="My LangChain Agent",
)

llm = ChatOpenAI(callbacks=[handler])
result = llm.invoke("Hello!")
print(handler.metrics)
```

## CLI Usage

```bash
export RORA_API_KEY=rora_xxx

agent-status status <agent_id>
agent-status run https://api.example.com/chat --prompts "Hello,How are you?"
agent-status list
agent-status register https://api.mycompany.com/chat --name "My Bot"
agent-status delete <agent_id>

# Private agent connector (portal token) — see "Private agents" above
agent-status expose \
  --agent-id <agent_uuid> \
  --token rtun_xxx \
  --target http://127.0.0.1:8080
```

## Gold Prompt Profiles

| Profile | Description |
|---------|-------------|
| `general` | Generic conversational prompts |
| `search_agent` | Web search and information retrieval |
| `code_generator` | Code generation and debugging |
| `data_retriever` | Database and API queries |
| `customer_support` | Support and FAQ handling |
| `creative_writer` | Content generation |

## Evaluation Types

| Type | Description |
|------|-------------|
| `basic` | Response format and latency checks |
| `llm_judge` | GPT-4 evaluates response quality |
| `all` | Both basic and LLM evaluation |

## Response Models

### Agent
```python
agent.id              # UUID
agent.name            # Display name
agent.endpoint_url    # HTTP endpoint (probe URL for tunnel agents)
agent.status          # active, paused, deleted
agent.last_status     # UP, DEGRADED, DOWN
```

### AgentStatus
```python
status.verdict       # UP, DEGRADED, DOWN, UNKNOWN
status.uptime_24h    # 24-hour uptime percentage
status.uptime_7d     # 7-day uptime percentage
status.latency_p50   # P50 latency (ms)
status.latency_p95   # P95 latency (ms)
status.pass_rate     # Pass rate (0-1)
status.total_checks  # Total probes run
```

### RunResult
```python
result.verdict          # UP, DEGRADED, DOWN
result.latency_p50      # P50 latency (ms)
result.latency_p95      # P95 latency (ms)
result.pass_rate        # Pass rate (0-1)
result.total_probes     # Probes sent
result.successful_probes  # Successful probes
result.by_region        # Per-region breakdown
result.judge_result     # LLM evaluation (if enabled)
```

## Error Handling

```python
from agent_status.client import AgentStatusError, AgentStatusAuthError, AgentStatusNotFoundError

try:
    status = agent_status.status("invalid-id")
except AgentStatusNotFoundError:
    print("Agent not found")
except AgentStatusAuthError:
    print("Invalid API key")
except AgentStatusError as e:
    print(f"Error: {e}")
```

## Environment Variables

| Variable | Description |
|----------|-------------|
| `RORA_API_KEY` | API key (CLI register/status/list) |
| `RORA_TUNNEL_TOKEN` | Tunnel connector token (`rtun_…`) for `agent-status expose` |
| `RORA_BASE_URL` | API base URL (optional, for testing) |

> Env vars keep the `RORA_` prefix for backward compatibility.

## Links

- [Homepage](https://agentstatus.dev)
- [Dashboard](https://platform.agentstatus.dev)
- [GitHub](https://github.com/Carmel-Labs-Inc/agent-status-sdk)

## License

MIT
