Metadata-Version: 2.5
Name: anyray-agent
Version: 0.1.2
Summary: Keeps an agent answering when its Anyray gateway is unreachable: an httpx2 transport for the OpenAI, Anthropic and LangChain SDKs that resends a call straight to the provider only when the gateway never handled it.
Project-URL: Homepage, https://docs.anyray.ai/enroll/agent-fallback-reference
License-Expression: MIT
Keywords: ai-gateway,anthropic,anyray,bedrock,fallback,langchain,llm,openai
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: httpx2<3,>=2.12
Provides-Extra: httpx
Requires-Dist: httpx<1,>=0.27; extra == 'httpx'
Description-Content-Type: text/markdown

# anyray-agent

Keeps an agent answering when its [Anyray](https://anyray.ai) gateway is unreachable. Give your SDK an `httpx2`
client built on `AnyrayTransport()`. While the gateway is up, every call goes through it unchanged. When the gateway
is down, a call goes straight to the provider instead.

```sh
pip install anyray-agent
```

Python 3.11 or later. One dependency, `httpx2`, which the current OpenAI and Anthropic SDKs already use.

## Use it

Keep the client pointed at your gateway, with your agent key, exactly as before. Add one argument.

```python
import os

import httpx2
from anthropic import Anthropic
from anyray_agent import AnyrayTransport

client = Anthropic(
    base_url=os.environ["ANYRAY_GATEWAY_URL"],
    api_key=os.environ["ANYRAY_CLIENT_KEY"],
    http_client=httpx2.Client(transport=AnyrayTransport()),
)
```

| Client | Where `AnyrayTransport()` goes |
| --- | --- |
| OpenAI SDK | `OpenAI(..., http_client=httpx2.Client(transport=AnyrayTransport()))` |
| Anthropic SDK | `Anthropic(..., http_client=httpx2.Client(transport=AnyrayTransport()))` |
| LangChain `ChatOpenAI` | `ChatOpenAI(..., http_client=httpx2.Client(transport=AnyrayTransport()))` |
| LangChain `ChatAnthropic` | `use_fallback(ChatAnthropic(...), transport=AnyrayTransport())` |

`use_fallback` comes from `anyray_agent.langchain`: `ChatAnthropic` has no client argument, so it swaps the clients
the model builds. Async code uses `AsyncAnyrayTransport` with `httpx2.AsyncClient` (`http_async_client=` on
`ChatOpenAI`, `async_transport=` on `use_fallback`). An SDK still on `httpx` 0.x (openai < 3, anthropic < 2) takes
the same classes from `anyray_agent.httpx` (`pip install "anyray-agent[httpx]"`).

## The provider key

A call can only go direct with a provider key on the host: `OPENAI_API_KEY` and `ANTHROPIC_API_KEY` are read from the
environment, or pass them:

```python
AnyrayTransport(provider_keys={"openai": "…", "anthropic": "…"})
```

To send Anthropic calls to AWS Bedrock instead of api.anthropic.com, name the region. Credentials come from
`AWS_BEARER_TOKEN_BEDROCK` or the standard AWS variables, or pass `api_key` or `credentials`. For an IAM role or a
profile, pass boto3's credential chain:

```python
import boto3

AnyrayTransport(bedrock={"region": "us-east-1", "credentials": boto3.Session().get_credentials})
```

Without a key for a provider, its calls are never sent direct.

## When a call goes direct

- **Sent direct:** the connection to the gateway never opened (refused, DNS, connect timeout) and the gateway's
  `/livez` check also fails. After the outage has lasted your org's fallback time (120 seconds by default), calls go
  direct without trying the gateway first. They return once the gateway has answered for your org's recover time
  (10 minutes by default).
- **Never sent direct:** a call that may already have reached the provider (a reset mid-request, any 5xx), so
  nothing is billed twice. Also a model or route only the gateway serves, and any call while your org has turned
  fallback off. Nothing goes direct until the agent has read that setting from the gateway.

The gateway key is never sent to a provider.

## See what it is doing

```python
transport = AnyrayTransport(on_status=lambda status: print(status.state))
transport.status()  # FallbackStatus(state="armed" | "direct" | "blocked" | "off", reason=..., since=..., calls_direct=...)
```

The agent also reports its state to the gateway, without any request content, so it shows on the Agents page.

Full reference: [docs.anyray.ai/enroll/agent-fallback-reference](https://docs.anyray.ai/enroll/agent-fallback-reference).

MIT licensed.
