---
phase: quick-260726-42c
plan: 01
subsystem: diagnostics
tags: [scripts, diagnostics, mac-address, arp, pep-723, rich]
requires: []
provides:
  - scripts/serial_mac_audit.py — serial→MAC correlation audit (Rich table + CSV)
affects: []
tech-stack:
  added: []
  patterns:
    - PEP 723 inline metadata with [tool.uv.sources] editable local pin
    - Console(stderr=True) for all Rich chrome so --csv pipes cleanly
key-files:
  created:
    - scripts/serial_mac_audit.py
  modified: []
decisions:
  - "ARP snapshot taken ONCE after all probes complete (not per-IP) — discovery unicast plus the two probe requests already populate the entries"
  - "Failed-probe rows keep verdict unknown even when an ARP entry exists, matching the locked verdict definition (unknown covers device-query failure); review IN-02 deliberately not applied"
  - "Strict IP/MAC shape guard (_is_valid_mac) on all ARP parsers, mitigating T-Q42C-01 beyond the locked regex"
  - "Conflicting duplicate ARP entries (same IP, different MACs) drop the IP entirely — an honest unknown beats a coin-flip verdict (review WR-04)"
  - "off-by-minus-one added as a fifth verdict class (magenta) so a -1 serial/MAC relationship is captured, not lumped into mismatch (review IN-01)"
metrics:
  duration: ~7 minutes (Tasks 1-2) + ~10 minutes review fixes; Task 3 pending
  completed: 2026-07-26
  tasks: 3/3
status: complete
---

# Quick Task 260726-42c: Serial/MAC correlation audit script Summary

**Standalone PEP 723 diagnostic that sweeps the LAN and correlates each LIFX device's serial, library-derived MAC, and independently observed ARP MAC into identical/off-by-one/mismatch/unknown verdicts grouped by product + firmware.**

## Status: COMPLETE — checkpoint signed off 2026-07-26

Tasks 1 and 2 executed and committed; all automated verification passed. **Task 3
(`checkpoint:human-verify`, gate=blocking) was run against the production fleet and
signed off by the operator on 2026-07-26.**

The sweep exposed two rendering defects, fixed in `0f5b228`:

- The progress bar set `total=len(tasks)` inside the discovery loop. Probes complete
  while `discover()` is still yielding, so `completed` momentarily equalled `total` and
  the bar snapped to 100% then fell back as the next device arrived — oscillating once
  per device. The task now stays indeterminate until discovery is exhausted.
- Every table column ellipsised (`d073d5…`, `identi…`), and the summary panel rendered
  both off-by columns as an identical `off-by-…`, erasing the distinction being
  measured. MACs now abbreviate to the final octet when the leading five match the
  serial, falling back to the full MAC on any earlier divergence.

A `rule_ok` grade and a "Library rule vs reality" panel were added: `verdict` grades the
serial→MAC relationship, which left the *library's* correctness visible only by
eye-diffing two columns.

### Finding

The audit did its job. 31 responders; `get_mac_address()` disagreed with the real ARP MAC
on 2 of them — LIFX Tiles on firmware **3.50**, whose real MAC equalled their serial while
the `version_major == 3` rule predicted serial + 1. Every other 3.x responder was on 3.90
and genuinely off-by-one.

The corrected rule came from LIFX via the operator -- vendor-stated, not inferred
from the sweep: the offset applies when
`version_major == 3 and version_minor >= 70`. Fixed in `f625afd` with a parametrised
boundary test; full suite 2625 passed.

### Outstanding checkpoint (Task 3)

On the fleet network (~73 production devices):

1. `uv run scripts/serial_mac_audit.py` — expect a transient progress bar on stderr,
   then a colour-coded table (green identical / yellow off-by-one / magenta
   off-by-minus-one / red mismatch / dim unknown) and the "Correlation by product +
   firmware" panel. Spot-check a known gen3 device: derived_mac should differ from
   serial in the final octet by +1 and the verdict should read off-by-one if the
   library rule is right for it.
2. `uv run scripts/serial_mac_audit.py --csv > /tmp/audit.csv` — progress still on
   stderr; `head /tmp/audit.csv` shows ONLY the header row and CSV data rows.
3. Optional: re-run once or twice — single rounds under-count (median 48/73); rows
   accumulate across runs when appended for analysis.

Resume signal: "approved", or describe issues (table rendering, CSV contamination,
wrong verdicts).

## Tasks completed

| Task | Name | Commits | Result |
| ---- | ---- | ------- | ------ |
| 1 | Pure correlation core + PEP 723 header + --self-test (TDD) | `1ada1f9` (RED), `120e949` (GREEN) | --self-test exits 0; all fixture assertions pass |
| 2 | Fleet sweep, ARP snapshot, Rich table + summary panel, --csv | `d3b4fcd` | --help exits 0 (PEP 723 env resolves rich 15.x + editable local lifx); self-test still green |
| — | Code-review fixes (260726-42c-REVIEW.md) | `6d9c161`, `16442c3`, `9d1cc02`, `f046db5` | All four warnings + IN-01/IN-03/IN-04 fixed; all gates re-run green |
| 3 | Human-verify checkpoint (real fleet sweep) | — | **PENDING — awaiting operator** |

## Code-review fixes (post-Task 2)

The coordinator's review (260726-42c-REVIEW.md, 4 warnings / 4 info) was addressed
before the fleet run — two warnings could silently corrupt the correlation data:

- **WR-04 (`6d9c161`)** — duplicate ARP IPs: lines that agree on the normalised MAC
  keep one entry, but an IP with CONFLICTING MACs is now dropped from the map so it
  degrades to an honest `unknown` instead of a coin-flip verdict. Shared
  `_add_arp_entry`/`_drop_conflicted` helpers cover all three parsers; self-test
  asserts both the agreeing- and conflicting-duplicate cases.
- **WR-02 + WR-03 (`16442c3`)** — `_run_snapshot` wraps the arp/ip subprocess with
  `timeout=10.0`, catches `FileNotFoundError`/`TimeoutExpired`, and degrades to an
  empty ARP map with a stderr warning instead of crashing after the sweep. Non-zero
  exits announce the command and exit status; dropped conflicted IPs are named on
  stderr.
- **WR-01 + IN-03 + IN-04 (`9d1cc02`)** — `_probe` appends the row and advances the
  progress bar BEFORE closing the connection; `close()` failures are contained (and
  logged) so a raising close can no longer silently drop a device. Non-`LifxError`
  failures now emit a diagnostic, and all interpolated exception text goes through
  `rich.markup.escape`.
- **IN-01 (`f046db5`)** — new `off-by-minus-one` verdict (magenta) for a real MAC of
  serial − 1 mod 256, with wraparound handling mirroring +1; flows into the table,
  summary panel, and self-test automatically via `VERDICT_STYLES`.
- **IN-02 — deliberately NOT applied:** the locked verdict definition
  (RESEARCH.md §Verdict computation) includes "device query failed" under `unknown`;
  the review itself calls the current behaviour defensible.

## What was built

- `scripts/serial_mac_audit.py` (single file, ~430 lines):
  - PEP 723 block pinning `rich>=15.0.0` and `lifx-async = { path = "../", editable = true }`
    — the self-test proves `lifx.__file__` resolves to the local checkout, not PyPI.
  - Pure core: `normalise_mac` (zero-pads unpadded macOS arp octets), `serial_to_mac`,
    `parse_arp_darwin` (captured line shapes, placeholder addresses: incomplete entry skipped, duplicate IP
    first-wins), `parse_arp_linux` + `parse_ip_neigh` (Linux bonus), `classify_verdict`
    (identical / off-by-one with `(o5+1) % 256` wraparound / mismatch / unknown).
  - Sweep: `discover(timeout, device_timeout=5.0, max_retries=3)`, probes started as
    devices arrive under `asyncio.Semaphore(16)`; exactly two requests per device
    (`get_version()` + `get_host_firmware()`, derived MAC from cache); failures isolated
    per device with the connection closed in `finally`.
  - Output: colour-coded Rich table + "Correlation by product + firmware" panel on
    stdout in human mode; stdlib `csv.writer(sys.stdout)` only in `--csv` mode; all
    progress/warnings on `Console(stderr=True)` in both modes.

## Verification results

Re-run in full after the review fixes:

- `uv run scripts/serial_mac_audit.py --self-test` → exit 0, all assertions pass
  (now including agreeing/conflicting ARP duplicates and off-by-minus-one cases)
- `uv run scripts/serial_mac_audit.py --help` → exit 0
- `uv run ruff format --check` + `uv run ruff check` → clean
- `git diff --quiet pyproject.toml uv.lock` → untouched (zero-dependency surface intact)
- Pre-commit hooks (incl. bandit, codespell) passed on every commit
- All commits GPG-signed (`git log --format="%G?"` → G) with DCO sign-off

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 2 - Missing critical functionality] Bandit B404 nosec for the subprocess import**
- **Found during:** Task 2 commit (pre-commit bandit hook failed)
- **Issue:** The plan anticipated B603/B607 nosec markers but bandit also flags B404
  (the `import subprocess` line itself); additionally, prose inside `# nosec` comments
  is parsed by bandit as test IDs, producing warnings.
- **Fix:** Bare `# nosec B404` / `# nosec B603 B607` markers with justification moved to
  adjacent comments, matching the threat-model mitigation for T-Q42C-01.
- **Files modified:** scripts/serial_mac_audit.py
- **Commit:** d3b4fcd

## Known Stubs

None — no placeholder data paths; the `--csv`/`--timeout` flags registered in Task 1
were fully wired in Task 2.

## Threat Flags

None — no security surface beyond the plan's threat model (fixed-argv subprocess calls
with strict IP/MAC parsing, as registered in T-Q42C-01).

## Self-Check: PASSED

- scripts/serial_mac_audit.py — FOUND
- Commit 1ada1f9 — FOUND
- Commit 120e949 — FOUND
- Commit d3b4fcd — FOUND
- Commit 6d9c161 — FOUND
- Commit 16442c3 — FOUND
- Commit 9d1cc02 — FOUND
- Commit f046db5 — FOUND
