Time-Series Collector agent
timeseries-collector — subscribes to MQTT (sensors, detections, Home Assistant state
changes, Sinergym) and writes the data to SQLite time-series tables for historical queries
and ML training. Writes are batched and auto-pruned by a retention window. No LLM.
Dependencies (installed on first spawn): aiomqtt.
Spawn
@catalog spawn timeseries-collector
It's an on-demand agent (not started at boot). Once running it collects continuously and is a natural companion to the Anomaly Detector.
Commands
@timeseries-collector stats # received/written counts and per-table row counts
@timeseries-collector prune # force a prune pass now
@timeseries-collector flush # flush buffered rows to SQLite now
Configuration
| Field | Meaning |
|---|---|
topics |
MQTT topic patterns to collect (default: sensors/#, detections, HA state, Sinergym) |
batch_interval |
seconds between SQLite flushes (default 5.0) |
batch_size |
buffered-row hint before a flush (default 200) |
retention_days |
auto-prune data older than N days (default 90) |
prune_interval_hours |
hours between prune passes (default 6.0) |
Returns total_received, total_written, buffer_sizes, table_rows, and retention_days.
Recording to Parquet, for the model loop
The collector keeps the dashboard's live window in SQLite. Readings the model loop
trains on and watches for drift go to Parquet instead, one file per source and hour
under the state directory's data/, written by the recorder library agent
(docs/design/data-storage.md):
@catalog spawn recorder {"topics": ["sensors/imu/#"], "source_field": "device"}
Or on a node, where it records what that node sees and the node ships each closed file to main:
{"type": "module", "target": "wactorz.agents.recorder_agent:RecorderAgent",
"name": "recorder", "node": "pi-kitchen",
"options": {"topics": ["sensors/imu/#"], "source_field": "device"}}
Each reading is a row of its source, the payload's source or device field, or the
topic without its wildcards when it names none, filed under its event time with the
arrival beside it. wactorz-data sources, wactorz-data profile <source> and
GET /api/data/{source}/profile say what was recorded; wactorz-data export writes a
window as one file for a notebook. The recorder needs the data extra (pyarrow).
What is recorded is what the trainer library agent trains on: spawned with a recipe,
a model's name and a source, it fits a new version on a window of the readings when
sent a task, and sets it as the model's shadow when it beats the version served
(docs/library.md, "Training a version"; docs/design/train-step.md).
And what the drift library agent watches: spawned with a model, and the source
when the served version does not say what it was trained on, it reads the last
day and the version's training window through the loader on a cadence, runs PSI
and KS on each feature and ADWIN on the model's recorded scores, and raises each
shift once, as an observation, a feed line and a message on
models/<name>/drift for a rule to retrain on (docs/library.md, "Watching for
drift"; docs/design/drift.md):
@catalog spawn drift {"model": "imu-detector", "source": "sensors/imu", "scores": "anomalies/imu"}