Metadata-Version: 2.1
Name: zidstats
Version: 0.1.0
Summary: Calibrated, distribution-free two-sample tests with a p-value and signed location/dispersion attribution (ZID / GPK / RISE / GET).
Author-email: Hao Chen <hxchen@ucdavis.edu>
License: MIT
Project-URL: Homepage, https://github.com/haochen6/zidstats
Keywords: two-sample-test,distribution-free,calibrated,p-value,kernel-test,graph-based-test,dispersion,attribution,nonparametric,high-dimensional
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy >=1.20
Requires-Dist: scipy >=1.6

# zidstats — calibrated, distribution-free two-sample tests (with a p-value)

A family of **two-sample tests** (ZID, GPK, RISE, GET) that, unlike a bare
distance/scalar, return a **calibrated p-value** and, by using *within-group*
similarity rather than only the cross term, stay sensitive to **dispersion/scale and
shape** differences — not just location — in high dimension. Each also exposes signed
`Z_W`/`Z_D` arms — one location-sensitive, one spread/shape-sensitive — for reading the
*direction* of a difference, not just its size (see Attribution below for details).

General-purpose: anywhere you compare two samples `X` and `Y` and want a *significance
statement* plus *how* they differ, not just a number.

**Dissimilarity-native:** the tests run on any pairwise dissimilarity — no coordinates, no
metric, no triangle inequality — so any object type works (sequences, graphs, distributions,
…). Pass point matrices with a `metric`, or a precomputed dissimilarity via
`two_sample_from_dissimilarity`; point inputs are just the convenience case.

## Install
```bash
pip install zidstats        # numpy, scipy only
```

## Quick start — one call, zero choices
```python
import numpy as np
from zidstats import two_sample_test

rng = np.random.default_rng(0)
X = rng.standard_normal((300, 64))
Y = rng.standard_normal((300, 64)) * 0.85       # same mean, 15% less spread
res = two_sample_test(X, Y)                      # default = ZID omnibus; nothing else to pick
print(res)
# ZID: p=0.00498 (stat=181.4, alpha=0.05) -> reject H0.
# Significant difference driven by SMALLER dispersion (Z_D=-12.4).
res.pvalue, res.statistic, res.reject, res.verdict
res.extra["members"]                       # member p-values: RISE, GPK@median, GPK@0.3med
res.extra["Z_D"], res.extra["Z_D_pval"]    # spread/shape-sensitive arm (from the RISE member)
res.extra["Z_W"], res.extra["Z_W_pval"]    # location-sensitive arm — clean two-axis (from RISE)
```
**Run this first.** The default is **ZID**, a broad-coverage omnibus over three members —
**RISE**, **GPK at the median bandwidth**, and **GPK at a small (0.3×median) bandwidth** (`GPK@0.3med` in `extra["members"]`). Its p-value comes from
a **permutation**: the members are permuted and flat-Simes-combined, and *that combined value is
then permuted too*, so the final p-value is the permutation p of the combination, not an analytic
Simes threshold (exact under exchangeability; `omnibus_null="simes"` uses the analytic threshold instead). It stays sensitive across location, dispersion, shape, and
matched-moment multimodality (the small-bandwidth member catches equal-mean/equal-variance
multimodal differences the others miss). It returns one calibrated p-value plus its RISE
member's signed **Z_W**/**Z_D** arms — location- and spread/shape-sensitive (see Attribution) —
and `extra["members"]` holds the three member p-values.

**Attribution (`Z_W`, `Z_D`).** `X` and `Y` are interchangeable — the p-value is the same
either way; only the *sign* of `Z_D` depends on order (swapping flips it), so put the reference
first for a directional read. `Z_D` is the spread/shape arm, not a variance test: for a pure
scale change `Z_D<0` reads as `Y` less dispersed than `X`, but `Z_D` also fires on shape, tail,
and matched-moment / multimodal differences — it can be significant, in either sign, even when
the first two moments of `X` and `Y` already match. `Z_W` is the location arm, but that reading
is clean only in moderate-to-high dimension: at low `d` a spread/shape difference also inflates
`Z_W`, so a large `Z_W` there does not by itself isolate a mean shift. Both are attribution
only; the omnibus p-value does not depend on the split.

**How the ZID p-value is built** (`omnibus_null="perm"`, the default): under one shared set of
`n_perm` label permutations, (1) each member statistic gets a permutation-rank p, (2) the three
are flat-Simes-combined into one value, and (3) *that combined value is re-ranked against the same
permutations* — step (3) is the second permutation, and it's what makes the whole thing exact
under exchangeability (no Simes/PRDS assumption). `omnibus_null="simes"` stops after an analytic
Simes threshold on the member p-values (faster; Type-I holds empirically). For a fully analytic
GPK, use `method="gpk", gpk_null="analytic"` (the fast fGPK approximation).

For a quick, fully analytic single-member check — or on very large `n` — use
**`method="rise"`**: RISE alone is itself a χ²₂ = `Z_W²+Z_D²` test, robust across geometries,
deterministic, no permutation.

## Routing — which member when
You almost never need to choose by hand: the default ZID omnibus covers everything. When you do
want a specialist (known geometry, max power, or discrete data), route on geometry rather than on
the smallest p-value:

| your situation | route → | why |
|----------------|---------|-----|
| **anything / unsure** (default) | **ZID** (omnibus) | broad coverage: joint permutation over RISE + GPK@median + GPK@0.3med; location / dispersion / shape / multimodality |
| **quick check / very large n** | **RISE** (`method="rise"`) | fast analytic single-member test (χ²₂), deterministic, no permutation |
| non-Euclidean / **manifold** / graph features | **RISE** | local rank-graph; a global kernel bandwidth smooths manifold density and goes blind to dispersion |
| plain **Euclidean / Gaussian**, want max power | **GPK** | kernel power specialist (`Z_W²+Z_D²`), strong on location *and* dispersion in high-d |
| need **attribution** (which arm?) | **RISE** (read `Z_W` & `Z_D`) | RISE/GET/GPK give a clean two-axis read |
| **discrete / heavy ties** (genotypes, counts, rounded) | **GET-discrete** (`method="get", discrete=True`) | validated tied-distance null; the graph/kernel nulls miscalibrate on ties |
| location said to dominate (`Z_D≈0`) | a **mean test** (t / Hotelling) | the verdict flags this; use zidstats when scale/shape may *also* differ |

## Non-Euclidean / object data
Euclidean is the default. To use another distance on point inputs, pass `metric=` — any scipy
`cdist` metric name or a callable `f(u, v) -> float`; the test then runs DIRECTLY on that
dissimilarity (no embedding, no metric assumption):
```python
res = two_sample_test(X, Y, metric="cosine")      # or "cityblock", "braycurtis", ...
```
If your data is only a **precomputed pairwise dissimilarity matrix** (graphs, trees, sequences,
any custom measure), use `two_sample_from_dissimilarity(D, n_x)` — it runs the family directly on
`D` (first `n_x` rows = group 1). The dissimilarity need not be a metric (no triangle inequality).

## Member roles
- **ZID — the default (omnibus).** Run it first: a joint permutation over RISE + GPK@median + GPK@0.3med → one calibrated p-value with the signed `Z_W`/`Z_D`. Broad coverage — location, dispersion, shape, matched-moment multimodality. The one call most users ever make.
- **RISE — the fast single-member path.** `method="rise"`: analytic χ²₂, deterministic, no permutation, manifold-aware, best-calibrated single test. Use for speed or very large `n`.
- **GPK — the Euclidean power specialist.** Use when the geometry is plainly Euclidean/Gaussian and you want maximum power on a location *or* continuous-dispersion difference. Caveat: a single global bandwidth can go blind on manifolds.
- **GET — the reference member.** The earliest of the family (edge-count); GPK and RISE descend from its within-group idea. A balanced graph alternative.
- **GET-discrete — the heavy-ties handler.** The validated path for discrete/tied data (genotypes 0/1/2, low counts, rounded values) where continuous nulls break.

## Methods
| method | type | p-value | sensitive to | tuning (default) |
|--------|------|---------|--------------|------------------|
| `zid`  | omnibus (default) | **two-pass permutation (members permuted → flat-Simes → the combination permuted too)** (`omnibus_null="perm"`; `"simes"` = analytic Simes over members (RISE analytic χ²₂; GPK members per `gpk_null`)) | location + dispersion + shape + multimodality | `sigma`=median, small=0.3×; `n_perm`=200 |
| `gpk`  | kernel | **permutation** (exact) | location + dispersion (Euclidean) | bandwidth = median |
| `gpk` + `gpk_null="analytic"` | kernel | analytic fGPK (fast approx of `gpk`; anti-conservative small-n) | location + dispersion | bandwidth = median |
| `rise` | graph-induced rank | **analytic χ²₂** | balance + mode-drop + dependence | k = ⌊N^0.65⌋ |
| `get`  | MST edge-count | **analytic χ²₂** | scale + dependence | k = ⌊N^0.5⌋ |

The family tests are **distribution-free** — validated Python ports of the CRAN packages
`kerTests` / `GraphRankTest` / `gTests`.

**Attribution arms.** RISE and GET are χ²₂ = `Z_W²+Z_D²` with both arms `N(0,1)`, so they give
a **clean two-axis read**: two orthogonal arms, `Z_W` (location-sensitive) and `Z_D`
(spread/shape-sensitive), each with an analytic p-value (`2·Φ(−|·|)`). For the kernel member,
`GPK = Z_W²+Z_D²` too (Song & Chen 2024): its `Z_D` arm is `N(0,1)` with an analytic p, while
the non-normal `Z_W` arm (unweighted `Z_{W,1.0}` ≈ standardized MMD) gets an exact
**permutation** p (`extra["Z_W_pval_perm"]`). fGPK also exposes the weighted `N(0,1)` arms
`ZW1`/`ZW2`.

### Note on "kernel" (≠ MMD)
GPK is a kernel test but **not** MMD: MMD compares the two samples through the **cross** term
and is location-dominated / scale-blind in high dimension; GPK combines the **within-group**
similarities and so detects dispersion collapse (`MMD` is the `Z_W` arm alone; GPK adds `Z_D`).

## Cite
Methods: GPK (Song & Chen 2024, *Biometrika* 111(3):755–770), RISE (Zhou & Chen 2023,
COLT/PMLR 195), GET (Chen & Friedman 2017, *JASA* 112(517):397–409).
