Metadata-Version: 2.4
Name: sgnmon
Version: 0.1.0
Summary: Monitoring and metrics for SGN streaming pipelines
Project-URL: Homepage, https://git.ligo.org/greg/sgnmon
Project-URL: Documentation, https://greg.docs.ligo.org/sgnmon
Project-URL: Repository, https://git.ligo.org/greg/sgnmon.git
Project-URL: Issues, https://git.ligo.org/greg/sgnmon/issues
Author-email: Olivia Godwin <olivia.godwin@ligo.org>
Maintainer-email: Olivia Godwin <olivia.godwin@ligo.org>
License-Expression: LGPL-3.0-or-later
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.11
Requires-Dist: gpstime
Requires-Dist: prometheus-client>=0.20
Requires-Dist: sgn-ts>=0.11
Requires-Dist: sgn>=0.11
Requires-Dist: starlette>=0.37
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn>=0.30
Provides-Extra: dev
Requires-Dist: httpx; extra == 'dev'
Requires-Dist: markdown-callouts>=0.2; extra == 'dev'
Requires-Dist: markdown-exec>=0.5; extra == 'dev'
Requires-Dist: mkdocs-coverage>=0.2; extra == 'dev'
Requires-Dist: mkdocs-gen-files>=0.3; extra == 'dev'
Requires-Dist: mkdocs-literate-nav>=0.4; extra == 'dev'
Requires-Dist: mkdocs-material-igwn; extra == 'dev'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'dev'
Requires-Dist: mkdocs>=1.3; extra == 'dev'
Requires-Dist: mkdocstrings[python]; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: mypy-extensions; extra == 'dev'
Requires-Dist: pip; extra == 'dev'
Requires-Dist: playwright; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: toml>=0.10; extra == 'dev'
Provides-Extra: docs
Requires-Dist: markdown-callouts>=0.2; extra == 'docs'
Requires-Dist: markdown-exec>=0.5; extra == 'docs'
Requires-Dist: mkdocs-coverage>=0.2; extra == 'docs'
Requires-Dist: mkdocs-gen-files>=0.3; extra == 'docs'
Requires-Dist: mkdocs-literate-nav>=0.4; extra == 'docs'
Requires-Dist: mkdocs-material-igwn; extra == 'docs'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'docs'
Requires-Dist: mkdocs>=1.3; extra == 'docs'
Requires-Dist: mkdocstrings[python]; extra == 'docs'
Requires-Dist: toml>=0.10; extra == 'docs'
Provides-Extra: lint
Requires-Dist: mypy; extra == 'lint'
Requires-Dist: mypy-extensions; extra == 'lint'
Requires-Dist: pip; extra == 'lint'
Requires-Dist: ruff; extra == 'lint'
Provides-Extra: test
Requires-Dist: httpx; extra == 'test'
Requires-Dist: playwright; extra == 'test'
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

<h1 align="center">sgnmon</h1>

<p align="center">Monitoring and metrics for SGN streaming pipelines</p>

<p align="center">
  <a href="https://git.ligo.org/greg/sgnmon/-/pipelines/latest">
    <img alt="ci" src="https://git.ligo.org/greg/sgnmon/badges/main/pipeline.svg" />
  </a>
  <a href="https://git.ligo.org/greg/sgnmon/-/pipelines/latest">
    <img alt="coverage" src="https://git.ligo.org/greg/sgnmon/badges/main/coverage.svg" />
  </a>
  <a href="https://greg.docs.ligo.org/sgnmon">
    <img alt="documentation" src="https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat" />
  </a>
  <a href="https://pypi.org/project/sgnmon/">
    <img alt="pypi version" src="https://img.shields.io/pypi/v/sgnmon.svg" />
  </a>
</p>

---

Observe [SGN](https://greg.docs.ligo.org/sgn/) pipelines from the outside:
collect Prometheus metrics (gap/non-gap content, latencies, throughput,
end-of-stream state) from running pipelines, serve them over HTTP, and
evaluate pipeline health checks suitable for container orchestration and
monitoring plugins.

## Resources

* [Documentation](https://greg.docs.ligo.org/sgnmon)
* [Source Code](https://git.ligo.org/greg/sgnmon)
* [Issue Tracker](https://git.ligo.org/greg/sgnmon/-/issues)

## Installation

```
pip install sgnmon
```

## Features

* **Non-invasive taps** -- attach probes to any existing pipeline's pads
  without modifying the graph, at three observation points per element
  (frames produced, frames received, and post-audioadapter consumption)
  so latency can be attributed per element
* **Monitoring elements** -- a pass-through `MonitorTransform` and terminal
  `MonitorSink` for explicit in-graph instrumentation
* **Prometheus metrics** -- frame counts, gap/data seconds, sample
  throughput, latency histograms, per-pad execution time, freshness
  timestamps, EOS state, and opt-in process CPU/memory metrics
* **Custom metrics** -- declare application metrics on your own elements
  (`MetricSpec`/`MetricsMixin`) or create families directly on the
  monitor; an `EventLatency` element measures per-event latency for
  event streams
* **Windowed statistics** -- rates, gap fractions, and latencies
  aggregated natively over a 5s...6h trailing-window ladder, for
  sub-minute resolution and consumers without PromQL
* **Health checks** -- a configuration-free default set (data freshness,
  throughput, latency drift) that self-paces its startup, plus opt-in
  absolute-latency, gap-fraction, and end-of-stream checks, aggregated
  into a JSON report
* **Built-in webserver** -- `/metrics`, `/health` (HTTP 503 when
  unhealthy), `/status`, `/graph`, and live updates over server-sent
  events, served from a background thread
* **Live dashboard** -- a self-contained web page drawing the pipeline
  graph with live per-element rates, latencies, gap fractions, and health
  state; composed elements expand in place to show their internal
  elements
* **CLI** -- `sgnmon run -- <app> ...` attaches monitoring to any sgn
  application with zero code changes; `sgnmon demo` runs a monitored
  example pipeline; `sgnmon check <url>` queries a running pipeline's
  health with plugin-friendly exit codes

## Quickstart

### Monitor any sgn application, zero code changes

```
$ sgnmon run --port 9090 -- sgn-cal --config config.yaml --model model.npz
[sgnmon] dashboard: http://localhost:9090/
```

### Tap an existing pipeline

```python
from sgnmon import Monitor, MonitorServer

monitor = Monitor()
monitor.tap(pipeline)  # observe every pad, no graph changes

with MonitorServer(monitor, host="0.0.0.0", port=9090):
    pipeline.run()
```

Default health checks (data freshness, throughput, latency drift) are
registered out of the box; tune or extend them with
`monitor.add_default_checks(...)`. Then point Prometheus at
`http://<host>:9090/metrics`, or check health by hand:

```
$ sgnmon check http://localhost:9090
```

### Instrument with elements

```python
from sgnmon import Monitor, MonitorTransform

monitor = Monitor()
mon = MonitorTransform(name="mon", sink_pad_names=["H1"], monitor=monitor)
pipeline.insert(src, mon, snk)
pipeline.link({
    "mon:snk:H1": "src:src:H1",
    "snk:snk:H1": "mon:src:H1",
})
```

### Try the demo

```
$ sgnmon demo --port 9090
dashboard: http://localhost:9090/
metrics:  http://localhost:9090/metrics
health:   http://localhost:9090/health
status:   http://localhost:9090/status
```

Open the dashboard in a browser to watch the pipeline graph update live.
