---
description: Read the live dashboard — the five panels, what a dash means, and how to tell a stalled run from a slow one.
---

# Watch a run live

```console
$ matmelt monitor
```

A read-only dashboard that refreshes in place. It never writes: the store is
opened with SQLite's `query_only` pragma, so watching a run cannot disturb it,
and the same command works on a finished workspace. Ctrl-C leaves with exit `0`.

```console
$ matmelt monitor --interval 5      # refresh every 5 seconds
$ matmelt monitor --once            # one frame, then exit (scripts, logs)
$ matmelt monitor --max-ticks 20    # a bounded watch
```

## The overview

```text
╭─ Run overview ──────────────────────────────────────────────────────────────╮
│ Run              nacl-demo                                                  │
│ Elapsed          25m 26s                                                    │
│ Total materials  39                  ██░░░                                  │
│ Done             4 (10.3%)         ░     ░                                  │
│ In flight        0                 ░ 10% ░                                  │
│ Failed           3                 ░     ░                                  │
│ Throughput       —                   ░░░░░                                  │
│ ETA              —                                                          │
│ Stop stage       (full funnel)                                              │
│                                                                             │
│ ███████████████████████████████████████                                     │
│ █ STRUCTURED 28 (72%)  █ SCREENED 2 (5%)  █ DONE 4 (10%)  █ EXPANDED 2 (5%) │
╰─────────────────────────────────────────────────────────────────────────────╯
```

The bar is the whole population by state, so you can see at a glance where
materials are piling up. `EXPANDED` is not a failure — it is a fan-out parent
that finished its job.

## The stage pipeline

```text
┃Stage       ┃ Backend          ┃  Ready ┃  Running ┃  Done ┃  Failed ┃ Speed ┃
│compose     │ laptop (local)   │      0 │        0 │    38 │       0 │     — │
│structure   │ laptop (local)   │      0 │        0 │    37 │       0 │     — │
│screen      │ laptop (local)   │     26 │        2 │     6 │       3 │     — │
│extract     │ laptop (local)   │      2 │        0 │     4 │       0 │     — │
```

**This is the stall diagnostic.** A large `Ready` with zero `Running`, holding
still, is never a science problem — it is a backend that is down, a dependency
that is missing, or a stage paused by an unhealthy health check. Go to
{doc}`doctor`. A small `Ready` with steady `Running` is simply a slow stage.

`Speed` is completions per minute once there are enough of them to mean
anything.

## Running now

```text
╭─ Running now (2) ───────────────────────────────────────────────────────────╮
┃Material                   ┃ Stage       ┃ Backend      ┃  Elapsed ┃  Try ┃ CPU ┃ GPU ┃
│Cl1Na1_3D-83               │ screen      │ laptop       │       7s │   #1 │ —   │ —   │
│Cl1Na1_3D-89               │ screen      │ laptop       │       7s │   #1 │ —   │ —   │
╰─────────────────────────────────────────────────────────────────────────────╯
```

`Try #2` or higher means this material has already faulted once — useful early
warning that something is wrong before it exhausts its budget.

**A dash means unknown, never zero.** In the frame above both jobs had just been
sighted for the first time, and a first sighting has no earlier sample to
measure against; later frames show percentages. A machine with no GPU shows
dashes in that column forever, and a remote is sampled at most every ten
seconds. CPU is a process-subtree figure, so a job using three cores honestly
reads above 100%.

Telemetry is best-effort by construction: it can fail or time out, and when it
does you get a dash. It can never take the dashboard down or slow the run.

## Completions and failures

```text
╭─ Recent completions ────────────────────────────────────────────────────────╮
│✓ Cl1Na1_3D-25                             │             5m 52s │   00:53:35 │
│✓ Cl1Na1_3D-16                             │             5m 52s │   00:53:35 │
╰─────────────────────────────────────────────────────────────────────────────╯
╭─ Recent failures ───────────────────────────────────────────────────────────╮
│Cl1Na1_3D-3      │ screen │ screened out: energy -578480.6344 eV/atom outsid… │
│Cl1Na1_3D-2      │ screen │ stage 'screen' cannot run on item 'Cl1Na1_3D-2': … │
╰─────────────────────────────────────────────────────────────────────────────╯
```

Reasons are truncated to fit. For the full text — and the attempt count that
tells a verdict from a fault — use `matmelt status`
({doc}`diagnose-a-failure`).

## Follow named materials

Swap the failures panel for a detail table on materials you care about:

```console
$ matmelt monitor --track Cl1Na1_3D-83,Cl1Na1_3D-1
$ matmelt monitor --track-file picked.txt
```

```text
╭─ Tracked materials ─────────────────────────────────────────────────────────╮
┃Material         ┃ State      ┃ Stage  ┃ Backend ┃ In state ┃ Info           ┃
│Cl1Na1_3D-83     │ STRUCTURED │ screen │ laptop  │       9s │ laptop:33ba286…│
│Cl1Na1_3D-1      │ FAILED     │ —      │ —       │  24m 35s │ stage 'screen' │
│Cl1Na1_3D-999    │ not found  │ —      │ —       │        — │ —              │
╰─────────────────────────────────────────────────────────────────────────────╯
```

An id that does not exist renders as **`not found`** rather than being dropped,
so a typo in your list is visible instead of silently ignored. Rows keep the
order you gave them.

## Related

- {doc}`../tutorials/t4-monitor` — the dashboard followed through a whole run.
- {doc}`diagnose-a-failure` — when the failures panel starts filling up.
- {doc}`stop-and-resume` — pausing the run you are watching.
