Metadata-Version: 2.4
Name: diwata-pulse-kit
Version: 0.1.1
Summary: Local-first telemetry collector + store for the Diwata stack.
Author: Shaznay Sison
License-Expression: AGPL-3.0-only
Project-URL: Homepage, https://github.com/Diwata-Labs/Pulse
Project-URL: Repository, https://github.com/Diwata-Labs/Pulse
Project-URL: Issues, https://github.com/Diwata-Labs/Pulse/issues
Keywords: telemetry,observability,events,fastapi,local-first
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.111
Requires-Dist: uvicorn>=0.29
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Dynamic: license-file

# Pulse

> **Status:** v0.1.0 in progress — collector + store + query CLI/JSON API + read-only dashboard are built (Phases 1–4); see `docs/working/backlog.md`.
> **Role (architecture):** Event topology backbone / "System events + telemetry" source of truth for the Diwata stack.
> **Stack:** Python 3.12+, FastAPI, SQLite (v0.1.0) → Postgres + Redis (later). Dashboard is a static vanilla-JS page served by Pulse (no node/build step).

---

## Thesis

Pulse is the **telemetry and observability layer for the Diwata stack**. Every product already
emits, or will emit, small typed events at meaningful moments — Grain emits `phase.close`,
`task.close`, `suggest.accept`, and `workflow.next.stop_reason` today; Assay will emit
`verification.completed` and check-level results next. Pulse is the place those events land,
are stored durably, and can be queried as metrics — so the stack can answer "how is the
workflow actually performing, across products, over time?" without each product re-inventing
storage and aggregation. Like the rest of the stack, Pulse is **local-first, opt-in, and
provider-neutral**: nothing is collected unless explicitly enabled, everything runs against a
local SQLite file before any hosted backend exists, and the ingest contract is a plain
versioned JSON event that any producer (or agent/familiar) can POST or drop on disk — no
vendor SDK, no proprietary wire format.

Pulse does not change how producers emit. Grain's `telemetry_service.emit` already POSTs to a
configured endpoint and falls back to `.grain/telemetry_queue.jsonl` when no endpoint is
reachable. Pulse simply **is** that endpoint (and the drain for those queue files). The
contract Grain shipped in v0.4.0 — `TelemetryEvent {event_type, version, timestamp, payload}` —
is the contract Pulse accepts, unchanged.

---

## What Pulse is

- **The collector** — an HTTP endpoint (and a file-drop drain) that accepts versioned
  `TelemetryEvent` JSON from any Diwata producer, exactly in Grain's shipped shape.
- **The store** — an append-only, durable record of every accepted event. Starts as a single
  local SQLite file; promotes to Postgres when a hosted deployment needs it.
- **The metrics surface** — a CLI (`pulse ...`) and JSON API that read events back as
  aggregates: event counts, per-type/per-product/per-phase rollups, stop-reason frequency,
  verification pass rates — so `grain metrics` and Assay reports can read cross-run history
  from Pulse instead of only local files.
- **Agent-usable / agent-agnostic** — a familiar can push events and query metrics headlessly
  over HTTP or the CLI with `--format json`. No interactive step, no browser, no SDK lock-in.
- **The dashboard** — a read-only, local-first **human** view over that same read API, served
  by Pulse itself at `GET /` (`pulse serve` serves it). A self-contained static page (vanilla
  JS, no build step) that renders the metrics as plain tables + inline bars — see below.
- **Opt-in and local-first** — default off at the producer (Grain's gate) and self-hostable on
  a developer's own machine. Nothing leaves the machine unless an endpoint is configured.

## What Pulse is NOT

- **Not a producer.** Pulse never instruments anyone's code. Producers own emission; Pulse
  owns transport, storage, and aggregation. (This is the boundary Grain's v0.4.0 contract
  states explicitly: "Grain only emits.")
- **Not a tracer / APM.** No distributed tracing, spans, flame graphs, or per-request latency
  profiling. Pulse ingests discrete domain events, not call stacks.
- **Not an error tracker.** It is not Sentry. Crash/exception capture is out of scope; events
  are intentional, typed, no-PII domain signals.
- **Not Chronicle.** Chronicle is the immutable **decision + rationale + high-risk action**
  audit trail. Pulse is operational **telemetry** (counts, durations, frequencies). They are
  distinct source-of-truth owners in the architecture.
- **Not Lore.** Pulse does not store normalized entity data (persons, companies, signals).
  It stores events about how the systems run.
- **Not a vendor-locked telemetry SaaS.** No proprietary wire format, no required client
  library. The contract is plain versioned JSON over HTTP or a `.jsonl` file drop.

---

## Where Pulse fits

| Layer | Owner |
|-------|-------|
| Workflow events (`phase.close`, `task.close`, `suggest.accept`, `workflow.next.stop_reason`) | **Grain** emits |
| Verification events (`verification.completed`, check results) | **Assay** emits (planned) |
| Transport · storage · aggregation · query | **Pulse** owns |
| System events + telemetry (source of truth) | **Pulse** (per `docs/canonical/architecture.md`) |

See `docs/working/v0.1.0_plan.md` for the v0.1.0 scope, `docs/working/integration_grain_assay.md`
for the producer-side contract, and `docs/working/backlog.md` for the phased build order.

---

## Dashboard

Pulse ships a read-only **dashboard** — the **human** view over the exact same read API that
agents/familiars consume as JSON. It is a self-contained static page (vanilla JS + CSS, **no
node, no build step, no framework**); it only `fetch`es the existing read endpoints and renders
them as plain tables with simple inline bars (no chart library). It adds **no** business logic —
every number comes from `GET /v1/metrics/*` and `GET /v1/events`.

It is served by Pulse itself, so running the collector serves the dashboard:

```bash
pulse serve                 # collector + dashboard at http://127.0.0.1:8006/
pulse dashboard             # print the dashboard URL (convenience over `pulse serve`)
pulse dashboard --open      # ...and open it in the default browser
```

Open `http://127.0.0.1:8006/` to see:

- **Overview** — event counts by type and by source (`/v1/metrics/counts`).
- **Phases** — per-phase rollup: closes, tasks done, task closes, last-closed (`/v1/metrics/phases`).
- **Stop reasons** — `workflow.next.stop_reason` frequency (`/v1/metrics/stop-reasons`).
- **Verification** — Assay pass-rate, per-target trends, and over-time (`/v1/metrics/verification`).
  Lights up automatically once Assay emits `verification.completed`; shows an empty state until then.
- **Recent events** — the newest events (`/v1/events?limit=`).

The dashboard is **read-only** and, like the other read paths, stays **open even when an ingest
token is set** (`PULSE_INGEST_TOKEN` gates only ingest). The JSON API is unchanged — the
dashboard mounts at `/` (+ `/dashboard.css`, `/dashboard.js`, `/static/`) without touching any
`/v1/*` route. Empty/zero states render gracefully against a fresh store; an optional
auto-refresh toggle is off by default.
