pbcheck audit report

1. Header

pbcheck audit of 'rare_cohort_biopsy.h5ad'

pbcheck 0.1.0, generated 2026-09-01T12:00:00Z.

Status: complete

This is a pbcheck v0.1.0 audit of one stratum of one file. It measures how a naive per-cell test and a donor-pseudobulk test behave on this data under a donor-permutation null. It is run outside pbcheck's pre-registered Phase 0 protocol, is not a Phase 0 measurement, and makes no claim about any publication or about whether any reported result is true.

2. Read-out

Under donor permutation, with no true signal to find, the naive per-cell test calls a median of 41 genes (0.5% of the 4310-gene universe; Monte-Carlo SE 0.412) over 20 permutations, coarse because few distinct donor splits exist; on the real labels it calls 63. Both counts are corrected over the whole universe on their own (solo BH).

The naive arm's inflation factor lambda is 3.21 (IQR 0.44): above the band 0.9 to 1.1, which is the donor-pseudobulk arm's band, shown here to describe the naive number and not as the naive arm's own criterion.

The donor-pseudobulk arm's lambda is 1.02 (inside the band 0.9 to 1.1); its permutation false-positive rate is 0.041 (MC SE 0.014) and its floor a median of 1 genes; on the real labels it calls 5 genes, corrected across both arms together (paired BH).

3 donors in 'lupus_nephritis', 4 in 'healthy'; 20 distinct donor splits exist.

After the thin-donor filter (fewer than 10 cells or 1000 counts), 3 and 4 pseudobulk profiles remain.

Real-label calls over the permutation floor: the per-cell arm calls 1.54 times its own floor, both counts corrected over the whole universe on their own (solo BH). At fewer than 8 donors in a group these ratios are contaminated by the per-cell leak and describe this run alone.

On this file, with condition labels shuffled between donors and therefore no real signal to find, the per-cell test still calls a median of 41 of 4310 genes at FDR 0.05 (0.5%), corrected over the whole gene universe on its own (solo BH); on the real labels, corrected the same way, it calls 63. With fewer than 8 donors in a group, the ratio of the real-label count to that floor is contaminated by the per-cell leak at this donor count and is not interpretable on this run. The donor is the replication unit this design supports.

Amendment 3 declares an operating envelope for the pseudobulk arm on synthetic oracles. For each donor-variance point it states the donor count per group at which the power target is reached, and each row says what the committed grid supports there: a point measured on the grid, or a count derived or extrapolated from it.
sigma_donor 0.2: at least 4 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'not in the grid; Amendment 1 frontier only')
sigma_donor 0.35: at least 8 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.793 at 8v8 (calibrated) -> n* <= 8')
sigma_donor 0.5: at least 13 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.486 at 12v12, the largest n tested -> n* > 12')
sigma_donor 0.7: at least 23 donors per group (power at least 0.6 at log2FC 1.0 in 200 genes; grid support 'ebayes power 0.003 at 8v8 -> n* far above 8')
The arm's calibration was evaluated at one hard regime (sigma_donor 0.5, 8 against 8 donors) and nowhere else. pbcheck does not estimate sigma_donor for real data, so whether this stratum lies inside that envelope is not determined here.

Everything in this report is a diagnostic of this file: the naive arm's inflation factor and permutation floor, the pseudobulk arm's inflation factor, floor and false-positive rate as its negative control, and the bookkeeping of the shared gene universe. None of it is a Phase 0 result, and pbcheck does not rate this file against the Phase 0 decision rule.

At least one group has fewer than 8 donors: the permutation null has few distinct donor splits and the floors are coarse. Amendment 5 Change 2 of pbcheck's protocol admits floor-based quantities outside the envelope only when every group has at least 8 donors, and this run is below that in at least one group, so its floor-based numbers describe this run alone. Do not compare them with a run on another file.

Only 20 distinct donor splits exist for this design, so the permutation null has 20 draws, not the 1000 requested; the floor's Monte-Carlo standard error (0.63) is correspondingly large.

3. What these words mean

lambda: the genomic inflation factor: the ratio of observed test statistics to the null expectation, with 1.0 meaning no inflation.

permutation floor: the number of genes a test calls under donor permutation, when condition labels carry no real signal; it is the test's own false-positive baseline on this data.

donor-permutation null: the null distribution built by reassigning condition labels between donors, keeping every donor's cells together, and rerunning the test on each reassignment.

gene universe: the fixed set of genes both arms are tested and corrected over, frozen before either arm sees the real labels.

replication unit: the unit whose independent draws the statistics assume; for donor data that unit is the donor, not the cell.

thin-donor filter: the rule that drops a donor's pseudobulk profile when it is built from too few cells or too few counts, rather than keeping a noisy profile.

operating envelope: the region of donor count and donor-to-donor variability that Amendment 3 declares for the pseudobulk arm on synthetic data; each of its points states the donor count per group at which the power target is reached, on the committed grid or by the derivation the grid support of that point names.

sigma_donor: a knob of pbcheck's synthetic simulator for how much donors differ from each other; it cannot be measured on your data, which is why the envelope question is left open.

Monte-Carlo SE: the standard error of a quantity estimated from a finite number of permutations; it shrinks as more permutations are drawn.

BH convention (solo vs paired): solo BH corrects an arm's p-values over the whole gene universe on its own; paired BH corrects both arms together over the genes common to both, so their real-label counts are directly comparable.

4. Design audit

No batch columns were provided.

5. Counts check

Examples of the values checked: 0.0, 1.0, 3.0, 12.0.

6. Naive per-cell arm

The paired floor is not shown: the pseudobulk arm left 812 genes without a value, so a paired correction over the two arms would not cover the same genes; the solo floor above stands alone.

Machinery check of the permutation engine, not a criterion of any kind: the inflation factor of the empirical permutation p-values is 1.02.

7. Donor-pseudobulk arm

Machinery check of the permutation engine, not a criterion of any kind: the inflation factor of the empirical permutation p-values is 1.01.

8. Shared universe bookkeeping

Builder rule: a gene is kept when its total count is at least 15 in at least half the pseudobulk profiles of each group, after the thin-donor filter

9. Settings and provenance

The engine was measured on one synthetic oracle point (sigma_donor 0.5, 8 against 8 donors, 1500 genes), recorded in pilot/gate/synthetic_gate_2026-08-15.json; that measurement is not repeated on this file. Of the settings below, only the ones listed as protocol constants are pre-registered values taken from pbcheck.gate_config (alpha, lambda_band, min_universe_size, min_cells, min_counts); the permutation counts, the universe filter parameters, the fallback universe rule and the display settings are the tool's own and are not protocol values.

10. Footer

Generated by pbcheck 0.1.0. Protocol: docs/PHASE0_SPEC.md and docs/AMENDMENTS.md in the pbcheck repository.