Metadata-Version: 2.4
Name: ceteris
Version: 0.2.0
Summary: Capture benchmark run environment fingerprints and gate comparisons between them.
Author: Anshuman Agrawal
License-Expression: MIT
Project-URL: Homepage, https://github.com/iemAnshuman/ceteris
Project-URL: Issues, https://github.com/iemAnshuman/ceteris/issues
Keywords: benchmark,hpc,reproducibility,mpi,slurm,provenance
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Benchmark
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-benchmark>=4; extra == "dev"
Dynamic: license-file

# ceteris

[![ci](https://github.com/iemAnshuman/ceteris/actions/workflows/ci.yml/badge.svg)](https://github.com/iemAnshuman/ceteris/actions/workflows/ci.yml)

**Wrap any benchmark. Refuse the comparison unless it is valid.**

A comparison of two benchmark numbers is valid if, and only if:

1. the only things that differ between the runs are the things you *meant* to vary, and
2. the difference is bigger than the noise.

Every benchmark harness -- hyperfine, Google Benchmark, JMH, pytest-benchmark,
criterion, MLPerf, OSU -- does some of (2) and none of (1). Their own docs list
the confounds and tell *you* to handle them: CPU governor, turbo, SMT, a stale
build, a different library version, one node in sixteen on another driver.
ceteris captures them, checks both conditions, and exits non-zero when either
fails. It sits underneath the harness you already use.

Named for *ceteris paribus*: all other things being equal.

```sh
pipx install git+https://github.com/iemAnshuman/ceteris.git
```

Python 3.9+. No runtime dependencies.

## Quickstart

Two real runs on the machine this README was written on, an Apple M4 laptop.
Compress a 27 MB file with `gzip -6` and `gzip -1`, each timed by hyperfine,
each repeated three times. No `--metric` needed: ceteris recognises hyperfine
and reads its export.

```console
$ ceteris run --label gzip-6 --repeats 3 -- hyperfine -N --warmup 1 --runs 5 'gzip -6 -c /tmp/payload.txt'
$ ceteris run --label gzip-1 --repeats 3 -- hyperfine -N --warmup 1 --runs 5 'gzip -1 -c /tmp/payload.txt'

$ ceteris compare
6 runs compared. Declared varying: nothing

MEASUREMENTS (per configuration):
  configuration    n  metric                  min     median        max  spread
  gzip-6    3  hyperfine.median_s     0.6401     0.6411     0.6417     0%
  gzip-6    3  hyperfine.min_s       0.639     0.6396     0.6411     0%
  gzip-1    3  hyperfine.median_s       0.51     0.5302     0.5328     4%
  gzip-1    3  hyperfine.min_s      0.5096     0.5265     0.5301     4%

NOISE FLOOR:
  hyperfine.median_s signal        gap 21% exceeds the largest within-configuration spread of 4%
  hyperfine.min_s  signal        gap 21% exceeds the largest within-configuration spread of 4%

UNDECLARED DIFFERENCES (comparison is not valid):
  execution.program_args  -N, --warmup, 1, --runs, 5, gzip -6 -c /tmp/p… (gzip-6 x3)  vs  -N, --warmup, 1, --runs, 5, gzip -1 -c /tmp/p… (gzip-1 x3)

DIFFERS, NOT GATING (informational severity):
  execution.command       hyperfine -N --warmup 1 --runs 5 gzip -6 -c /… (gzip-6 x3)  vs  hyperfine -N --warmup 1 --runs 5 gzip -1 -c /… (gzip-1 x3)
  system.load_1m          5 distinct values across 6 runs (range 2.57 to 2.71)

Matched on 112 other fields.
$ echo $?
1
```

The tool found a real 21% signal above a 4% noise floor -- and still refused,
because the command line changed and nobody said that was the experiment.
Declare it, and ask for a certificate:

```console
$ ceteris compare --vary execution.program_args --require-signal --certify
6 runs compared. Declared varying: execution.program_args

MEASUREMENTS (per configuration):
  configuration    n  metric                  min     median        max  spread
  gzip-6    3  hyperfine.median_s     0.6401     0.6411     0.6417     0%
  gzip-6    3  hyperfine.min_s       0.639     0.6396     0.6411     0%
  gzip-1    3  hyperfine.median_s       0.51     0.5302     0.5328     4%
  gzip-1    3  hyperfine.min_s      0.5096     0.5265     0.5301     4%

NOISE FLOOR:
  hyperfine.median_s signal        gap 21% exceeds the largest within-configuration spread of 4%
  hyperfine.min_s  signal        gap 21% exceeds the largest within-configuration spread of 4%

DECLARED VARYING (expected):
  execution.program_args  -N, --warmup, 1, --runs, 5, gzip -6 -c /tmp/p… (gzip-6 x3)  vs  -N, --warmup, 1, --runs, 5, gzip -1 -c /tmp/p… (gzip-1 x3)

DIFFERS, NOT GATING (informational severity):
  execution.command       hyperfine -N --warmup 1 --runs 5 gzip -6 -c /… (gzip-6 x3)  vs  hyperfine -N --warmup 1 --runs 5 gzip -1 -c /… (gzip-1 x3)
  system.load_1m          5 distinct values across 6 runs (range 2.57 to 2.71)

Matched on 112 other fields.

OK: every difference was declared. Comparison is valid.

ceteris-certified v1 configs=2 n=3,3 vary=execution.program_args waive= strict=0 verdict=ok noise=4% sha256:b3969923f4e69dd904dfc11239aa4d6a719b67ca64434beb732e76aa13276969
$ echo $?
0
```

That last line is portable. Paste it into a README, a pull request or a
paper's artifact appendix, and anyone holding the records can check it:

```console
$ ceteris verify 'ceteris-certified v1 configs=2 n=3,3 vary=execution.program_args waive= strict=0 verdict=ok noise=4% sha256:b3969923f4e69dd904dfc11239aa4d6a719b67ca64434beb732e76aa13276969' examples/hyperfine/*.json
ceteris: verified: ok
```

The six records are committed under [`examples/hyperfine/`](examples/hyperfine/).

## The other failure mode: a result that is not a result

An MPI ping-pong ([`examples/pingpong.c`](examples/pingpong.c)) built at `-O3`
and at `-O0`, three runs each, declaring the flags as the variable:

```console
$ ceteris compare examples/pingpong/*.json --vary build.cxx_flags
6 runs compared. Declared varying: build.cxx_flags

MEASUREMENTS (per configuration):
  configuration    n  metric                  min     median        max  spread
  tuned-O3    3  bandwidth_gbs         48.59      49.15       52.9     9%
  tuned-O0    3  bandwidth_gbs          45.5      46.51      51.57    13%

NOISE FLOOR:
  bandwidth_gbs    WITHIN NOISE  gap 6% between configuration medians is not larger than the 13% spread within a single configuration

CONFOUNDED WITH A DECLARED VARIABLE (re-run; do not waive):
  execution.program moves in lockstep with build.cxx_flags:
      build.cxx_flags = -O0  ->  execution.program = ./pingpong_O0  (3 runs)
      build.cxx_flags = -O3  ->  execution.program = ./pingpong_O3  (3 runs)
  execution.program_sha256 moves in lockstep with build.cxx_flags:
      build.cxx_flags = -O0  ->  execution.program_sha256 = d82a3febc7d215d4d0c0d2d…  (3 runs)
      build.cxx_flags = -O3  ->  execution.program_sha256 = 56d7060c5d6ea948be9d66f…  (3 runs)

UNDECLARED DIFFERENCES (comparison is not valid):
  execution.program         ./pingpong_O3 (tuned-O3 x3)  vs  ./pingpong_O0 (tuned-O0 x3)
  execution.program_sha256  56d7060c5d6ea948be9d66f482b7614b0240ffde7b4a3… (tuned-O3 x3)  vs  d82a3febc7d215d4d0c0d2d0d1293416a62b0b4adcd5c… (tuned-O0 x3)

DECLARED VARYING (expected):
  build.cxx_flags           -O3 (tuned-O3 x3)  vs  -O0 (tuned-O0 x3)

DIFFERS, NOT GATING (informational severity):
  execution.command         mpirun -n 2 ./pingpong_O3 1048576 200 (tuned-O3 x3)  vs  mpirun -n 2 ./pingpong_O0 1048576 200 (tuned-O0 x3)
  system.load_1m            2.96 (tuned-O3 x2, tuned-O0 x3)  vs  2.78 (tuned-O3)

Matched on 126 other fields.
$ echo $?
1
```

Three things happened there, and each is a class of invalid comparison that
gets published:

- **WITHIN NOISE.** The 6% gap between the builds is smaller than the 13%
  scatter inside one build. Two numbers is not a measurement.
- **CONFOUNDED.** The binary hash moves in lockstep with the declared
  variable. Of course it does -- they are different builds -- but that is
  exactly what a stale build looks like too, and ceteris cannot tell the
  difference. It asks you to declare it rather than waive it.
- **`system.power_source = battery`** is in every one of these records. This
  laptop was unplugged. That field compares equal here because it was equal
  on every run; it would not have been equal against a run from last week.

## What it captures

One record per run, ~131 fields, every field carrying its own provenance and one
of four states. This is [`examples/pingpong/20260827-215234Z-tuned-O3-2.json`](examples/pingpong/20260827-215234Z-tuned-O3-2.json),
excerpted to the 41 fields that carried a value plus a few that degraded:

```json
{
  "fields": {
    "build.compiler_id": {
      "p": "mpicc --version",
      "s": "value",
      "v": "clang"
    },
    "build.compiler_path": {
      "p": "--compiler",
      "s": "value",
      "v": "/opt/homebrew/bin/mpicc"
    },
    "build.compiler_version": {
      "p": "mpicc --version",
      "s": "value",
      "v": "22.1.8"
    },
    "build.cxx_flags": {
      "p": "--cxx-flags",
      "s": "value",
      "v": "-O3"
    },
    "execution.command": {
      "p": "wrapped command line",
      "s": "value",
      "v": "mpirun -n 2 ./pingpong_O3 1048576 200"
    },
    "execution.launcher": {
      "p": "heuristic split of the wrapped command line",
      "s": "value",
      "v": "mpirun"
    },
    "execution.launcher_args": {
      "p": "heuristic split of the wrapped command line",
      "s": "value",
      "v": [
        "-n",
        "2"
      ]
    },
    "execution.program": {
      "p": "heuristic split of the wrapped command line",
      "s": "value",
      "v": "./pingpong_O3"
    },
    "execution.program_args": {
      "p": "heuristic split of the wrapped command line",
      "s": "value",
      "v": [
        "1048576",
        "200"
      ]
    },
    "execution.program_sha256": {
      "p": "sha256 of ./pingpong_O3",
      "s": "value",
      "v": "56d7060c5d6ea948be9d66f482b7614b0240ffde7b4a392a4c78b5eb9ccf62b7"
    },
    "execution.workdir": {
      "p": "os.getcwd()",
      "s": "value",
      "v": "/Users/anshumanagrawal/codes/workplace/2026_projects/ceteris"
    },
    "hardware.arch": {
      "p": "platform.uname().machine",
      "s": "value",
      "v": "arm64"
    },
    "hardware.cpu_cores_logical": {
      "p": "sysctl -n hw.logicalcpu",
      "s": "value",
      "v": 10
    },
    "hardware.cpu_cores_physical": {
      "p": "sysctl -n hw.physicalcpu",
      "s": "value",
      "v": 10
    },
    "hardware.cpu_model": {
      "p": "sysctl -n machdep.cpu.brand_string",
      "s": "value",
      "v": "Apple M4"
    },
    "hardware.gpu_models": {
      "d": "nvidia-smi not on PATH",
      "p": "nvidia-smi",
      "s": "not_applicable"
    },
    "hardware.hostnames": {
      "p": "platform.uname().node",
      "s": "value",
      "v": [
        "MacBook-Air-7.local"
      ]
    },
    "hardware.kernel": {
      "p": "platform.uname().release",
      "s": "value",
      "v": "24.6.0"
    },
    "hardware.node_count": {
      "p": "single-host capture",
      "s": "value",
      "v": 1
    },
    "hardware.os": {
      "p": "platform.uname().system",
      "s": "value",
      "v": "Darwin"
    },
    "hardware.os_version": {
      "p": "platform",
      "s": "value",
      "v": "15.7.7"
    },
    "packs.active": {
      "p": "pack activation",
      "s": "value",
      "v": [
        "hpc",
        "python"
      ]
    },
    "runtime.env.OMP_NUM_THREADS": {
      "d": "unset",
      "p": "$OMP_NUM_THREADS",
      "s": "not_applicable"
    },
    "runtime.mpi_implementation": {
      "p": "mpirun --version",
      "s": "value",
      "v": "Open MPI"
    },
    "runtime.mpi_launcher": {
      "p": "shutil.which('mpirun')",
      "s": "value",
      "v": "/opt/homebrew/bin/mpirun"
    },
    "runtime.mpi_version": {
      "p": "mpirun --version",
      "s": "value",
      "v": "5.0.9"
    },
    "scheduler.job_id": {
      "d": "not running under Slurm",
      "p": "$SLURM_JOB_ID",
      "s": "not_applicable"
    },
    "source.branch": {
      "p": "git branch --show-current",
      "s": "value",
      "v": "main"
    },
    "source.commit": {
      "p": "git rev-parse HEAD",
      "s": "value",
      "v": "9c98a7b8a1b02cab930ecd27594502d6aa5d45bc"
    },
    "source.dirty": {
      "p": "git status --porcelain",
      "s": "value",
      "v": true
    },
    "source.repo_path": {
      "p": "--repo",
      "s": "value",
      "v": "/Users/anshumanagrawal/codes/workplace/2026_projects/ceteris"
    },
    "system.aslr": {
      "p": "darwin policy",
      "s": "value",
      "v": "always-on"
    },
    "system.container": {
      "p": "darwin",
      "s": "value",
      "v": "no"
    },
    "system.cpu_governor": {
      "d": "darwin does not expose this",
      "p": "sysfs",
      "s": "not_applicable"
    },
    "system.load_1m": {
      "p": "os.getloadavg()[0]",
      "s": "value",
      "v": 2.96
    },
    "system.power_source": {
      "p": "pmset -g batt",
      "s": "value",
      "v": "battery"
    },
    "system.smt": {
      "p": "sysctl hw.logicalcpu vs hw.physicalcpu",
      "s": "value",
      "v": "off"
    },
    "system.swap": {
      "p": "sysctl -n vm.swapusage",
      "s": "value",
      "v": "in-use"
    },
    "system.thermal_throttle": {
      "p": "pmset -g therm",
      "s": "value",
      "v": "none"
    },
    "system.virtualization": {
      "p": "sysctl -n kern.hv_vmm_present",
      "s": "value",
      "v": "none"
    },
    "toolchain.python": {
      "p": "python3 --version",
      "s": "value",
      "v": "3.14.7"
    }
  },
  "meta": {
    "captured_at": "2026-08-27T21:52:34+00:00",
    "content_hash": "5b3cdc03723ca49d901834c23075cdb0a525e36d26c561c4ad232b4f18c7c042",
    "kind": "run",
    "label": "tuned-O3",
    "repeat": 2,
    "schema_version": 2,
    "series": "tuned-O3@2026-08-27T21:52:34+00:00",
    "tool": "ceteris",
    "tool_version": "0.1.0"
  },
  "metrics": {
    "bandwidth_gbs": {
      "p": "regex /bandwidth ([0-9.]+) GB/s/",
      "s": "value",
      "v": 48.585
    }
  },
  "run": {
    "drift": [],
    "duration_s": 0.072,
    "exit_code": 0,
    "output": "size 1048576 bytes  iters 200  bandwidth 48.585 GB/s",
    "output_truncated": false,
    "started_at": "2026-08-27T21:52:34+00:00"
  }
}
```

| namespace | what |
|---|---|
| `source` | commit, dirty flag, submodules |
| `build` | compiler, flags, CMake cache entries |
| `execution` | launcher, launcher args, program, program args, **sha256 of the binary** |
| `system` | governor, turbo, SMT, ASLR, hugepages, power source, thermal throttle, load, container, VM |
| `hardware` | CPU, NUMA, GPU model/count/driver, CUDA -- **for every node of an allocation** |
| `runtime` | MPI implementation and version, transport configuration, tuning variables |
| `toolchain`, `deps` | language toolchain versions, **lockfile hashes**, container image |
| `parallelism`, `scheduler` | ranks, threads, binding intent, Slurm allocation |

`runtime.env.*`, `toolchain.*` and `deps.*` come from **ecosystem packs** that
activate from what is in the tree and on PATH: `Cargo.toml` turns on
`RUSTFLAGS` and `Cargo.lock`; `mpirun` on PATH turns on LCI, UCX and OMPI
variables; `nvidia-smi` turns on NCCL and CUDA. A Node project is never asked
about UCX.

### Four states, not two

| state | meaning |
|---|---|
| `value` | captured |
| `not_applicable` | structurally absent -- no GPU in this machine |
| `unknown` | may exist, could not be read -- `nvidia-smi` hung |
| `error` | the collector itself failed |

Two GPU-less laptops agreeing on `not_applicable` is a match. A laptop
against a GPU node is a difference. An `unknown` anywhere is neither: it is
reported separately and the comparison is not certified, because the response
differs -- a difference means the experiment is confounded, an unknown means
the capture is incomplete.

The rule: **a tool's absence implies the thing's absence only when the tool
*is* the thing.** No `nvidia-smi` means no NVIDIA stack. No `git` does not
mean no commit.

```console
$ ceteris compare examples/run-a.json examples/run-b-gpu-probe-failed.json
2 runs compared. Declared varying: nothing

UNKNOWN (could not be captured -- comparison is not certified):
  hardware.gpu_driver
      run-b-gpu-probe-failed: unknown: timed out after 10.0s
      run-a: known, <not applicable>

Matched on 107 other fields.
$ echo $?
2
```

(`run-b-gpu-probe-failed.json` is `run-a.json` with one field hand-edited to
`unknown`; a hung `nvidia-smi` cannot be produced on a machine without one.
It is the only non-captured file in `examples/`.)

## Harnesses it understands

| harness | how it is recognised | what ceteris does |
|---|---|---|
| hyperfine | `hyperfine` on the command line | adds `--export-json` if absent, reads median and min |
| Google Benchmark | any `--benchmark_*` flag | adds `--benchmark_out`, reads `real_time` |
| pytest-benchmark | `pytest` with `--benchmark*` | adds `--benchmark-json`, reads medians |
| JMH | `-rf json` | reads `-rff` output |
| criterion | `cargo bench` | reads fresh `target/criterion/*/new/estimates.json` |
| OSU micro-benchmarks | `osu_*` | parses the size/value table |
| nccl-tests | `*_perf` | parses out-of-place busbw |
| MLPerf loadgen | `mlperf` / `loadgen` in the command | reads `mlperf_log_summary.txt` |

Anything else: `--metric NAME='regex with one (group)'`, or `--ingest FILE`.
A pattern that does not match records `unknown`, never zero. The hyperfine,
Google Benchmark and pytest-benchmark parsers are tested against files those
tools produced; the other five against formats reconstructed from their
documentation, and say so in their tests.

## Where it plugs in

**Any script.** `ceteris run -- <command>` passes the command's exit code
through, so wrapping changes nothing.

**A batch job.** Inside the sbatch script, wrapping the launcher. Every node of
the allocation is fingerprinted via a one-task-per-node `srun` inside the
existing allocation; a node that fails to report makes the hardware fields
`unknown`, because fifteen of sixteen nodes is not a fingerprint.

```bash
#SBATCH -N 16 -n 256
export LCI_ATTR_PACKET_SIZE=73728
ceteris run --label "lci-$LCI_ATTR_PACKET_SIZE" --repo ~/hpx --cmake-cache ~/hpx/build \
    --store ~/campaigns/collectives -- srun ./bench_all_to_all --size 4096
```

**A pull request.** The action runs the benchmark on base and head in one job
and fails the check unless the comparison is valid:

```yaml
- uses: iemAnshuman/ceteris@main
  with:
    command: hyperfine -N 'target/release/mytool bench.dat'
    repeats: "5"
    require-signal: "true"
```

**pytest.** `pytest --ceteris` records the session; with pytest-benchmark
installed, its results come along.

**A library.** `from ceteris import compare`, `from ceteris.runner import
run_command`.

## Checking the capture itself

A fingerprint is 150 fields, and reading it by eye to decide whether ceteris
understood the machine does not scale. `ceteris doctor` does it:

```console
$ ceteris doctor
this machine: 127 fields (29 captured, 98 not applicable, 0 unknown, 0 error)

WORTH KNOWING (will widen the noise floor):
  system.power_source  battery: on battery the CPU is throttled
  source.dirty         the working tree had uncommitted changes; the commit does not describe the code
```

Three groups, and the middle one is the point:

- **LOOKS WRONG** -- a claim contradicted by the machine itself. "No GPU" is
  a normal answer on a laptop and a suspicious one where `/dev/kfd` exists.
  Exit 1. This is the check that would have caught the AMD bug before a
  cluster did.
- **COULD NOT BE READ** -- every unknown, with its reason. Exit 2.
- **WORTH KNOWING** -- turbo on, a scaling governor, battery power, a busy
  machine, a dirty tree. Never a failure; these are the things that make
  numbers move before any code changes.

Pass a record file to inspect a capture from another machine, which is the
useful form when someone sends you one:

```sh
ceteris doctor examples/rostam/het2.json
```

## Declaring intent

```sh
--vary build.cxx_flags                  # exact
--vary 'runtime.env.LCI_*'              # glob
--vary execution.program_args           # the sweep is in the arguments; rank count still gates
--waive hardware.cpu_model:"same partition, different node draw"
--require-signal                        # exit 4 unless a metric beats the noise floor
--strict                                # informational fields gate too
```

`--waive` needs a reason and the reason lands in the certificate. A declared
field that did not vary is flagged (the sweep did not apply); a declaration
matching no field is flagged (typo). Fields carry a severity -- `critical`
and `material` gate, `informational` is shown -- and unlisted fields gate.

## Exit codes

| code | meaning |
|---|---|
| 0 | valid: every difference was declared |
| 1 | undeclared differences, or a confound |
| 2 | something could not be captured, or the environment changed mid-run |
| 3 | usage |
| 4 | valid but within noise (`--require-signal`) |

## Configuration

`./ceteris.toml` is picked up automatically:

```toml
packs = ["cuda"]                       # force a pack on

[capture]
env_allowlist = ["MY_PROJECT_TUNABLE"]

[severity]
"hardware.hostnames" = "critical"      # heterogeneous partition: the nodes matter

[metrics]
latency_us = "avg latency ([0-9.]+) us"
```

The record format is specified in [`docs/SPEC.md`](docs/SPEC.md) so that
harnesses can emit it directly.

## What it does not do

- It does not measure. The harness measures; ceteris asks where the result went.
- It does not claim statistical significance. Median and range, and the
  honest word *unassessed* below three repeats.
- It does not capture the transport MPI actually negotiated, per-rank
  observed affinity, or NUMA topology beyond a node count. Each could be
  filled with a plausible value, and a plausible value in a fingerprint is
  the exact error the tool exists to catch.

## Status

Alpha, but the parts that matter have run on real hardware. CI covers Ubuntu
and macOS on Python 3.9 through 3.14. Ten records captured on **LSU's Rostam
cluster** are committed under [`examples/rostam/`](examples/rostam/), each
verified against ground truth taken in the same job:

| verified on real hardware | |
|---|---|
| `ceteris run` with `--repeats` across two nodes | Open MPI 5.0.7, gcc 14.3.0 |
| per-node fan-out, homogeneous **and heterogeneous** | CUDA 12.8 via `nvcc` |
| Slurm field mapping | NVIDIA V100 via `nvidia-smi` |
| Linux CPU, NUMA, governor, turbo, SMT, ASLR, THP | AMD Instinct MI100 via `rocm-smi` |
| container identity inside a real Apptainer image | a real `CMakeCache.txt` |
| rust, go, node and JVM toolchains and lockfiles | |

Those runs found six bugs, all fixed: `ceteris compare` certified three runs
whose benchmark had exited 183; `system.*` was taken from the head node
alone; a failing `nvidia-smi` on a GPU-less login node blocked every
comparison; the ROCm parser asked `rocm-smi` two questions in one call and
lost every card row; `java -version` writes to stderr so no JDK was ever
recorded; and one fingerprint contradicted itself about being in a container.

**Still unverified — treat as unproven, not as working.** Every one of these
is written from documentation, which the ROCm case showed is wrong about half
the time:

- schedulers other than Slurm (PBS, LSF, Flux, Grid Engine)
- MPI other than Open MPI (MPICH, Intel MPI, MVAPICH)
- five of the eight harness adapters: JMH, criterion, OSU, nccl-tests, MLPerf
- ARM CPUs (`/proc/cpuinfo` has no `model name` there; it degrades to
  `unknown`, so it fails safe)
- the AMD `cpufreq/boost` turbo path, fan-out beyond two nodes, and GPUs
  within one node reporting different driver versions

If you run ceteris on any of those, `ceteris capture -o fp.json` and the
resulting file is the most useful thing you can send.

```sh
python -m venv .venv && .venv/bin/pip install -e '.[dev]' && .venv/bin/python -m pytest
```
