Metadata-Version: 2.3
Name: heywaffle
Version: 0.1.0
Summary: Webhook routing + Claude Agent SDK observability
License: MIT
Keywords: webhooks,agents,claude,anthropic,observability,inbox
Author: Stef van den Ham
Author-email: stef@mindthecode.com
Requires-Python: >=3.11,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Framework :: FastAPI
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries
Provides-Extra: hub
Requires-Dist: claude-agent-sdk (>=0.1.0)
Requires-Dist: croniter (>=2.0.0) ; extra == "hub"
Requires-Dist: fastapi (>=0.115.0) ; extra == "hub"
Requires-Dist: httpx (>=0.28.0)
Requires-Dist: httpx-sse (>=0.4.0)
Requires-Dist: jinja2 (>=3.1.0) ; extra == "hub"
Requires-Dist: sqlmodel (>=0.0.22) ; extra == "hub"
Requires-Dist: sse-starlette (>=2.0.0) ; extra == "hub"
Requires-Dist: uvicorn[standard] (>=0.32.0) ; extra == "hub"
Project-URL: Homepage, https://heywaffle.dev
Project-URL: Issues, https://github.com/Hyra/waffle/issues
Project-URL: Repository, https://github.com/Hyra/waffle
Description-Content-Type: text/markdown

# Waffle

Webhook routing + Claude Agent SDK observability. Write your agent as a standalone Python program; Waffle gives it an inbox (webhooks, cron, sync API), retries with dead-lettering, and a web UI that shows every Claude session it ran.

```
                                ┌─────────────────────────────────────────┐
                                │          Waffle hub (FastAPI)           │
external webhook  ──────────────▶ POST /webhooks/{source}                 │
sync API client   ──────────────▶ POST /api/invoke/{agent}                │
cron scheduler    ─── internal ─▶                                         │
                                │                                         │
                                │  inbox per agent  ─── SSE ──▶  agent    │
                                │  trace ingest     ◀──────────  agent    │
                                │                                         │
                                │  /ui/  dashboard + session views        │
                                └─────────────────────────────────────────┘
```

Agents live in their own projects, anywhere on disk. They depend on `waffle_sdk` (this repo) and run as their own OS process. The hub stays language-agnostic in spirit — it talks HTTP to the agents and SQLite to itself.

## Contents

1. [Quickstart — start the hub](#quickstart--start-the-hub)
2. [Build your first agent](#build-your-first-agent)
3. [Use case 1 — Develop locally with `once` mode](#use-case-1--develop-locally-with-once-mode)
4. [Use case 2 — Triggered by external webhooks](#use-case-2--triggered-by-external-webhooks)
5. [Use case 3 — Run on a schedule (cron)](#use-case-3--run-on-a-schedule-cron)
6. [Use case 4 — Synchronous request/response API](#use-case-4--synchronous-requestresponse-api)
7. [Inspecting what happened in the UI](#inspecting-what-happened-in-the-ui)
8. [Reference](#reference)

---

## Quickstart — start the hub

```bash
git clone https://github.com/you/waffle.git
cd waffle
poetry install --extras hub        # hub deps live behind the [hub] extra
poetry run waffle                  # → http://127.0.0.1:8765
poetry run waffle --reload         # dev mode with autoreload
poetry run waffle --port 9000      # custom port
```

> `poetry install` (no extras) pulls in just the SDK, which is what agents
> need. Add `--extras hub` when you want to run the web server too.

- UI: http://127.0.0.1:8765/ui/
- OpenAPI / Swagger: http://127.0.0.1:8765/docs
- SQLite database: `data/waffle.db` (override via `WAFFLE_DB_PATH`)

Leave the hub running in its own terminal for everything below.

---

## Build your first agent

We'll use the Recipe Converter as the running example: it takes a URL and returns a structured recipe by asking Claude to fetch the page.

```bash
mkdir -p ~/agents/recipe-converter
cd ~/agents/recipe-converter
poetry init --no-interaction --name recipe-converter --python ">=3.11,<4.0"
```

Edit `pyproject.toml`:

```toml
[project]
name = "recipe-converter"
version = "0.1.0"
requires-python = ">=3.11,<4.0"
dependencies = [
    "waffle (>=0.1.0,<0.2.0)",
]

[tool.poetry]
packages = [{ include = "recipe_converter", from = "src" }]

[tool.poetry.dependencies]
waffle = { path = "../../waffle", develop = true }

[tool.poetry.scripts]
recipe-converter = "recipe_converter.__main__:main"

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"
```

Write the handler in `src/recipe_converter/agent.py`:

```python
from claude_agent_sdk import (
    ClaudeAgentOptions, ClaudeSDKClient, ResultMessage,
)
from waffle_sdk import AgentContext

# Routing filter for inbox mode (use case 2). Source "recipe" is just a label
# — your real integration would use "whatsapp" or "linear" or similar.
SUBSCRIPTIONS = [{"source": "recipe", "filter": {}}]

RECIPE_SCHEMA = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "source_url": {"type": "string"},
        "ingredients": {"type": "array", "items": {"type": "string"}},
        "instructions": {"type": "array", "items": {"type": "string"}},
        "servings": {"type": ["string", "null"]},
        "prep_time": {"type": ["string", "null"]},
        "cook_time": {"type": ["string", "null"]},
    },
    "required": ["title", "source_url", "ingredients", "instructions"],
}

async def handle(item: dict, ctx: AgentContext) -> dict:
    url = item["url"]
    options = ClaudeAgentOptions(
        system_prompt="Extract a recipe from the URL using WebFetch.",
        tools=["WebFetch"],
        permission_mode="bypassPermissions",
        output_format={"type": "json_schema", "schema": RECIPE_SCHEMA},
    )
    async with ClaudeSDKClient(options=options) as client:
        await client.query(f"Extract the recipe at: {url}")
        async for msg in client.receive_response():
            if isinstance(msg, ResultMessage):
                return msg.structured_output or {}
    raise RuntimeError("no result from Claude")
```

Wire the CLI in `src/recipe_converter/__main__.py`:

```python
from waffle_sdk import run
from .agent import SUBSCRIPTIONS, handle

def main() -> None:
    run(handle, agent="recipe-converter", subscriptions=SUBSCRIPTIONS)

if __name__ == "__main__":
    main()
```

```bash
poetry install
```

Your agent now supports four modes of operation.

---

## Use case 1 — Develop locally with `once` mode

**When to use this:** writing the handler, iterating on prompts, debugging tool use. No hub required.

```bash
poetry run recipe-converter once \
  --input '{"url":"https://visitsweden.nl/te-doen/eten-drinken/recepten/vegan-meatballs-recept/"}'
```

The handler runs once with the parsed `--input` as `item`. Its return value is printed as pretty JSON on stdout. Failures print a stack trace to stderr and exit with code 1.

**Bonus: traces appear in the UI automatically.** If the hub at `http://127.0.0.1:8765` is reachable, the SDK tails the JSONL session file Claude Code writes and forwards each line to the hub. You'll see the session in the dashboard within a second of completion. To skip explicitly:

```bash
poetry run recipe-converter once --input '{...}' --no-trace
poetry run recipe-converter once --input '{...}' --hub https://other-hub.example.com
```

---

## Use case 2 — Triggered by external webhooks

**When to use this:** Linear / WhatsApp / GitHub / any external system pushes events. Waffle receives them, picks the right agent based on a filter, and queues the item for processing.

**T1 — hub** (already running from Quickstart)

**T2 — agent in inbox mode**
```bash
cd ~/agents/recipe-converter
poetry run recipe-converter inbox
```

On startup the agent:
1. Registers its `SUBSCRIPTIONS` with the hub (idempotent — safe to restart).
2. Opens an SSE connection to `/inbox/recipe-converter/stream`.
3. For each item event: claim → invoke handler → POST complete (or fail with stack trace).

**T3 — simulate the webhook**
```bash
curl -sX POST http://127.0.0.1:8765/webhooks/recipe \
  -H 'content-type: application/json' \
  -d '{"url":"https://visitsweden.nl/te-doen/eten-drinken/recepten/vegan-meatballs-recept/"}'
```

T2's log shows `processing item N (source=recipe)` → `completed item N`. Inspect the result:

```bash
curl -s 'http://127.0.0.1:8765/inbox/recipe-converter?status=completed' | jq '.[-1].result'
```

Or open http://127.0.0.1:8765/ui/agents/recipe-converter/inbox — the item is there with a link to its trace.

### How the URL maps to the agent

The `recipe` in `/webhooks/recipe` and the `"source": "recipe"` in `SUBSCRIPTIONS` must be the same string — that is the only thing connecting them. The hub doesn't know about agent names from the URL; it looks up subscriptions whose `source` field matches.

```
POST /webhooks/<source>          ─┐
                                  │ same string
SUBSCRIPTIONS = [{                │
    "source": "<source>",  ───────┘
    "filter": {...},
}]
```

You pick the string yourself — it can be `recipe`, `whatsapp`, `linear`, `inbound-mail`, anything. Pick names that match the upstream system that will be calling the URL.

The webhook response tells you whether routing actually matched:

```bash
curl -sX POST http://127.0.0.1:8765/webhooks/recipe \
  -H 'content-type: application/json' -d '{"url":"..."}' | jq
# {
#   "enqueued": [42],
#   "skipped_duplicates": [],
#   "matched_agents": ["recipe-converter"]    ← non-empty means routed
# }
```

`matched_agents: []` means no agent had a subscription that fits — your URL string doesn't match any registered `source`, or filters rejected the payload, and the item is dropped. To see what's currently registered:

```bash
curl -s http://127.0.0.1:8765/subscriptions | jq
```

### Sharing one webhook URL across multiple agents

A common reality: Linear sends *all* webhook events for *all* your projects to one URL. Waffle routes by filter, so each agent only sees what it asked for.

```python
# linear-triage agent: only DEVOPS team's Issues and Comments
SUBSCRIPTIONS = [{
    "source": "linear",
    "filter": {
        "data.team.key": "DEVOPS",
        "type": ["Issue", "Comment"],
    },
}]
```

Filter rules: dot-notation lookup, scalar value = equality, list value = "in". All keys must match (AND). One inbound webhook can fan out to several agents — each gets its own inbox copy and is deduped per-agent on `(source, idempotency_key, agent)`.

### Re-sending the same payload during testing

The hub deduplicates webhook items by hashing `(source, payload)` — this is what makes webhook retries from providers like Linear safe. It also means that if you test with the exact same `curl` twice, the second call returns `"skipped_duplicates": ["recipe-converter"]` and no new item is queued.

To re-trigger with the same payload, either add a unique `Idempotency-Key` header:

```bash
curl -sX POST http://127.0.0.1:8765/webhooks/recipe \
  -H 'content-type: application/json' \
  -H "Idempotency-Key: $(date +%s)-$RANDOM" \
  -d '{"url":"https://visitsweden.nl/..."}' | jq
```

…or use the sync API (Use case 4), which generates a fresh UUID idempotency key per call and returns the handler's result directly.

### Retries and dead-lettering

If the handler raises, the item goes back to `pending` and is re-pushed via SSE. After `MAX_ATTEMPTS` (default 3) the item moves to `deadletter` and stays visible in the UI with the last error.

---

## Use case 3 — Run on a schedule (cron)

**When to use this:** weekly check, daily report, periodic sync. The hub's scheduler turns cron expressions into inbox items — your agent doesn't need to know about time at all.

Declare the cron in your agent:

```python
def main() -> None:
    run(
        handle,
        agent="vinylify-devops",
        subscriptions=[],
        crons=[
            # Mondays 09:00 — payload is what the handler will receive as `item`
            {"schedule": "0 9 * * 1", "source": "cron",
             "payload": {"task": "weekly_audit"}},
        ],
    )
```

Start in inbox mode. The agent registers its crons (idempotent), and the hub's scheduler (ticks every 30s) enqueues items at the right moments. Your handler treats them like any other item.

For impatient testing without waiting for the next minute, register a cron via API and force a faster schedule:

```bash
curl -sX POST http://127.0.0.1:8765/crons/register \
  -H 'content-type: application/json' \
  -d '{
    "agent": "recipe-converter",
    "schedule": "* * * * *",
    "source": "recipe",
    "payload": {"url":"https://visitsweden.nl/te-doen/eten-drinken/recepten/vegan-meatballs-recept/"}
  }'
# within ~60s your inbox-mode agent receives an item
```

---

## Use case 4 — Synchronous request/response API

**When to use this:** scripts, REST clients, Slack slash commands — anything that needs the handler's result back immediately.

Same inbox pipeline as the other use cases. The difference: the hub holds your HTTP connection open until the agent posts complete (or fails out to deadletter).

**Prerequisite:** agent must be running in `inbox` mode (T1 + T2 from Use case 2).

```bash
curl -sX POST http://127.0.0.1:8765/api/invoke/recipe-converter \
  -H 'content-type: application/json' \
  -d '{"url":"https://visitsweden.nl/te-doen/eten-drinken/recepten/vegan-meatballs-recept/"}' \
  | jq

# {
#   "item_id": 17,
#   "result": { "title": "Veganistische gehaktballetjes ...", "ingredients": [...], ... }
# }
```

Add `?timeout=N` to override the default 300-second wait.

| code | meaning |
|------|---------|
| 200  | handler completed; body has `{item_id, result}` |
| 502  | handler failed `MAX_ATTEMPTS=3` times; body has `{item_id, attempts, last_error}` |
| 503  | no agent of that name is currently connected to the hub |
| 504  | agent did not complete within `?timeout=N` seconds (default 300) |

The 503 is an upfront check so you don't sit around waiting on a queue no one is reading.

### Calling from a browser (CORS + bearer token)

Same pattern as the curl example, but two env vars on the hub enable it from a web page:

```bash
WAFFLE_CORS_ORIGINS=https://app.example.com,http://localhost:3000 \
WAFFLE_API_TOKEN=some-long-random-string \
poetry run waffle
```

- `WAFFLE_CORS_ORIGINS` — comma-separated list of origins the hub will send CORS headers for. Leave unset for same-origin or curl-only use.
- `WAFFLE_API_TOKEN` — if set, `POST /api/invoke/{agent}` requires `Authorization: Bearer <token>` and returns 401 otherwise. Leave unset in dev / on localhost.

Both apply **only** to `/api/invoke/*` at the moment. Webhooks, agent endpoints, and the UI remain open (see security notes below).

Then in your web app:

```html
<script>
async function callAgent() {
  const r = await fetch("https://waffle.example.com/api/invoke/recipe-converter", {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "Authorization": "Bearer some-long-random-string",
    },
    body: JSON.stringify({ url: "https://visitsweden.nl/..." }),
  });
  if (!r.ok) throw new Error(await r.text());
  const { item_id, result } = await r.json();
  console.log(result);
}
</script>
```

**Caveat about the token in client-side JS.** Anything you put in a browser's JavaScript is visible to anyone who opens devtools. The bearer token here is a low-bar measure — it keeps casual scrapers and random traffic out, it does **not** protect you against a determined user of your web app. For stronger guarantees, put a backend in front (your own API that holds the real token and proxies to Waffle), or use short-lived per-user tokens. Rotate the token by restarting the hub with a new value if it leaks.

**`file://` origins.** Browsers send `Origin: null` for pages opened directly from disk. Either add `null` to `WAFFLE_CORS_ORIGINS`, or serve your HTML via `python -m http.server` and whitelist `http://localhost:8000`.

---

## Inspecting what happened in the UI

Open http://127.0.0.1:8765/ui/.

- **Dashboard** (`/ui/`) — recent sessions, recent inbox items grouped per agent, registered subscriptions and cron triggers. Auto-refresh 5s.
- **Agent inbox** (`/ui/agents/{agent}/inbox`) — full inbox table: status, attempts, claimed-by, last error, trace link. Auto-refresh 3s.
- **Session detail** (`/ui/sessions/{session_id}`) — three tabs:
  - **Conversation** — chat-bubble view, user + assistant text only.
  - **Transcript** — every meaningful event as a row: colored badge (`USER`, `THINKING`, `WEBFETCH`, …), one-line preview, tokens / duration / timestamp on the right.
  - **Debug** — same row layout but includes everything (system events, `ai-title`, `last-prompt`, per-turn `tokens` summary, …).

Click any row to open a sticky detail pane on the right. For `tool_use` events the pane shows the `INPUT` JSON and the paired `TOOL RESULT · {duration}` (or `ERROR · …` if the tool reported failure) together — no need to scroll back and forth.

Stats header at the top of every session detail: turn count, wall duration, total input/output tokens, raw line count.

---

## Reference

### Hub commands

```bash
poetry run waffle                          # default: 127.0.0.1:8765
poetry run waffle --port 9000              # custom port
poetry run waffle --reload                 # autoreload while editing hub code
WAFFLE_DB_PATH=/tmp/x.db poetry run waffle # custom DB location
```

### Environment variables

| Variable | Applies to | Purpose |
|---|---|---|
| `WAFFLE_DB_PATH` | hub | SQLite file location (default `data/waffle.db`) |
| `WAFFLE_CORS_ORIGINS` | hub | Comma-separated allowlist of origins for browser `fetch()`. Empty = CORS middleware not mounted. |
| `WAFFLE_API_TOKEN` | hub | If set, `POST /api/invoke/*` requires `Authorization: Bearer <token>`. Empty = no auth (dev). |
| `WAFFLE_HUB_URL` | agent | Hub URL the agent connects to (default `http://127.0.0.1:8765`). |
| `WAFFLE_RUNNER_NAME` | agent | Identifier used when claiming items (default: hostname). |

### Agent CLI

```bash
my-agent once --input '{"…":"…"}'                # dev/test, prints result on stdout
my-agent once --input '{…}' --no-trace           # skip trace forwarding
my-agent once --input '{…}' --hub https://…     # use a different hub
my-agent inbox                                   # connect to hub, process items
my-agent inbox --hub https://waffle.example.com  # custom hub URL
my-agent inbox --runner my-laptop                # identifier for this runner
WAFFLE_HUB_URL=https://… my-agent inbox          # env override of --hub
```

### Hub HTTP endpoints

| Path | Purpose |
|------|---------|
| `POST /webhooks/{source}` | External webhook ingest, routed via subscriptions |
| `POST /api/invoke/{agent}?timeout=N` | Sync request/response (waits for handler) |
| `POST /subscriptions/register` | Register `{agent, source, filter}` |
| `GET  /subscriptions` | List subscriptions |
| `POST /crons/register` | Register `{agent, schedule, source, payload}` |
| `GET  /crons` | List cron triggers |
| `GET  /inbox/{agent}` | List items (filter `?status=…`) |
| `GET  /inbox/{agent}/stream` | SSE stream of inbox items (used by SDK) |
| `POST /inbox/{agent}/claim/{id}` | Mark an item claimed by a runner |
| `POST /inbox/{agent}/complete/{id}` | Mark an item completed with `result` |
| `POST /inbox/{agent}/fail/{id}` | Mark an item failed with `error` |
| `POST /traces/{session_id}/events` | Trace ingest (SDK forwards JSONL lines) |
| `GET  /traces` / `GET /traces/{session_id}` | Inspect captured traces |
| `GET  /ui/...` | Web UI |
| `GET  /docs` | OpenAPI / Swagger UI |

### Architectural principles

- **Agents are self-contained programs.** `once` mode runs without the hub. Hub integration (inbox, traces, sync API) is opt-in.
- **The inbox is the universal primitive.** Webhooks, cron, and sync API all create inbox items; agents process them through the same handler.
- **Cron → inbox item, not cron → agent invocation.** Time triggers create items the same way webhooks do, so retries / dead-lettering / observability work the same.
- **No agent config UI.** Agents are code; subscriptions and cron triggers are declared in the agent and registered on startup.
- **Secrets split.** The hub holds inbound webhook signing secrets (when added). Agents hold outbound API tokens (Notion, Linear, …) in their own env.

### Project layout

```
waffle/
├── pyproject.toml
├── README.md
├── data/waffle.db                       # SQLite (gitignored)
└── src/
    ├── waffle_sdk/                      # depended on by every agent
    │   ├── runner.py                    # `once` and `inbox` CLI subcommands
    │   ├── trace.py                     # JSONL session file watcher
    │   └── types.py                     # AgentContext, Handler
    └── waffle/                          # the server
        ├── server.py                    # FastAPI app + endpoints
        ├── models.py                    # SQLModel: Subscription, InboxItem,
        │                                # CronTrigger, TraceEvent
        ├── pubsub.py                    # in-memory SSE fan-out + sync waiters
        ├── scheduler.py                 # cron tick loop
        ├── filtering.py                 # subscription filter DSL
        └── ui/
            ├── parsing.py               # JSONL → Conversation/Transcript/Debug
            ├── router.py                # Jinja2 routes
            └── templates/
```

Agents live outside this repo — see "Build your first agent" above for the full walkthrough.

