pbcheck audit report

1. Header

pbcheck audit of 'lupus_kidney_pilot.h5ad'

pbcheck 0.1.0, generated 2026-09-01T12:00:00Z.

Status: complete

This is a pbcheck v0.1.0 audit of one stratum of one file. It measures how a naive per-cell test and a donor-pseudobulk test behave on this data under a donor-permutation null. It is run outside pbcheck's pre-registered Phase 0 protocol, is not a Phase 0 measurement, and makes no claim about any publication or about whether any reported result is true.

2. Read-out

Under donor permutation, with no true signal to find, the naive per-cell test calls a median of 41 genes (0.5% of the 8213-gene universe; Monte-Carlo SE 0.63) over 1000 permutations; on the real labels it calls 63. Both counts are corrected over the whole universe on their own (solo BH).

The naive arm's inflation factor lambda is 3.21 (IQR 0.44): above the band 0.9 to 1.1, which is the donor-pseudobulk arm's band, shown here to describe the naive number and not as the naive arm's own criterion.

The donor-pseudobulk arm's lambda is 1.02 (inside the band 0.9 to 1.1); its permutation false-positive rate is 0.041 (MC SE 0.014) and its floor a median of 1 genes; on the real labels it calls 5 genes, corrected across both arms together (paired BH).

11 donors in 'lupus_nephritis', 10 in 'healthy'; 352716 distinct donor splits exist.

After the thin-donor filter (fewer than 10 cells or 1000 counts), 10 and 10 pseudobulk profiles remain.

Real-label calls over the permutation floor: the per-cell arm calls 1.54 times its own floor, both counts corrected over the whole universe on their own (solo BH). The donor-pseudobulk arm calls 5 times its floor, both counts corrected across the two arms together (paired BH).

On this file, with condition labels shuffled between donors and therefore no real signal to find, the per-cell test still calls a median of 41 of 8213 genes at FDR 0.05 (0.5%), corrected over the whole gene universe on its own (solo BH); on the real labels, corrected the same way, it calls 63. A gene list produced by a per-cell test on this data cannot be separated from that floor. The donor is the replication unit this design supports. The donor-pseudobulk test called 5 genes on the real labels against a permutation median of 1, both corrected across the two arms together over the genes they have in common (paired BH).

Amendment 3 declares an operating envelope for the pseudobulk arm on synthetic oracles. For each donor-variance point it states the donor count per group at which the power target is reached, and each row says what the committed grid supports there: a point measured on the grid, or a count derived or extrapolated from it.
sigma_donor 0.2: at least 4 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'not in the grid; Amendment 1 frontier only')
sigma_donor 0.35: at least 8 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.793 at 8v8 (calibrated) -> n* <= 8')
sigma_donor 0.5: at least 13 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.486 at 12v12, the largest n tested -> n* > 12')
sigma_donor 0.7: at least 23 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.003 at 8v8 -> n* far above 8')
The arm's calibration was evaluated at one hard regime (sigma_donor 0.5, 8 against 8 donors) and nowhere else. pbcheck does not estimate sigma_donor for real data, so whether this stratum lies inside that envelope is not determined here.

Everything in this report is a diagnostic of this file: the naive arm's inflation factor and permutation floor, the pseudobulk arm's inflation factor, floor and false-positive rate as its negative control, and the bookkeeping of the shared gene universe. None of it is a Phase 0 result, and pbcheck does not rate this file against the Phase 0 decision rule.

The design audit found a batch column that separates the two conditions ('batch'): condition and batch cannot be told apart in this stratum, and neither arm's real-label result can be read as a condition effect.

3. What these words mean

lambda: the genomic inflation factor: the ratio of observed test statistics to the null expectation, with 1.0 meaning no inflation.

permutation floor: the number of genes a test calls under donor permutation, when condition labels carry no real signal; it is the test's own false-positive baseline on this data.

donor-permutation null: the null distribution built by reassigning condition labels between donors, keeping every donor's cells together, and rerunning the test on each reassignment.

gene universe: the fixed set of genes both arms are tested and corrected over, frozen before either arm sees the real labels.

replication unit: the unit whose independent draws the statistics assume; for donor data that unit is the donor, not the cell.

thin-donor filter: the rule that drops a donor's pseudobulk profile when it is built from too few cells or too few counts, rather than keeping a noisy profile.

operating envelope: the region of donor count and donor-to-donor variability that Amendment 3 declares for the pseudobulk arm on synthetic data; each of its points states the donor count per group at which the power target is reached, on the committed grid or by the derivation the grid support of that point names.

sigma_donor: a knob of pbcheck's synthetic simulator for how much donors differ from each other; it cannot be measured on your data, which is why the envelope question is left open.

Monte-Carlo SE: the standard error of a quantity estimated from a finite number of permutations; it shrinks as more permutations are drawn.

BH convention (solo vs paired): solo BH corrects an arm's p-values over the whole gene universe on its own; paired BH corrects both arms together over the genes common to both, so their real-label counts are directly comparable.

4. Design audit

5. Counts check

Examples of the values checked: 0.0, 1.0, 3.0, 12.0.

6. Naive per-cell arm

Machinery check of the permutation engine, not a criterion of any kind: the inflation factor of the empirical permutation p-values is 1.02.

7. Donor-pseudobulk arm

Machinery check of the permutation engine, not a criterion of any kind: the inflation factor of the empirical permutation p-values is 1.01.

8. Shared universe bookkeeping

Builder rule: a gene is kept when its total count is at least 15 in at least half the pseudobulk profiles of each group, after the thin-donor filter

9. Settings and provenance

The engine was measured on one synthetic oracle point (sigma_donor 0.5, 8 against 8 donors, 1500 genes), recorded in pilot/gate/synthetic_gate_2026-08-15.json; that measurement is not repeated on this file. Of the settings below, only the ones listed as protocol constants are pre-registered values taken from pbcheck.gate_config (alpha, lambda_band, min_universe_size, min_cells, min_counts); the permutation counts, the universe filter parameters, the fallback universe rule and the display settings are the tool's own and are not protocol values.

10. Footer

Generated by pbcheck 0.1.0. Protocol: docs/PHASE0_SPEC.md and docs/AMENDMENTS.md in the pbcheck repository.