zidstats 0.1.0

Calibrated, distribution-free two-sample tests with a p-value and signed location/dispersion attribution (ZID / GPK / RISE / GET).

zidstats — calibrated, distribution-free two-sample tests (with a p-value)

A family of two-sample tests (ZID, GPK, RISE, GET) that, unlike a bare distance/scalar, return a calibrated p-value and, by using within-group similarity rather than only the cross term, stay sensitive to dispersion/scale and shape differences — not just location — in high dimension. Each also exposes signed Z_W/Z_D arms — one location-sensitive, one spread/shape-sensitive — for reading the direction of a difference, not just its size (see Attribution below for details).

General-purpose: anywhere you compare two samples X and Y and want a significance statement plus how they differ, not just a number.

Dissimilarity-native: the tests run on any pairwise dissimilarity — no coordinates, no metric, no triangle inequality — so any object type works (sequences, graphs, distributions, …). Pass point matrices with a metric, or a precomputed dissimilarity via two_sample_from_dissimilarity; point inputs are just the convenience case.

Install

pip install zidstats        # numpy, scipy only

Quick start — one call, zero choices

import numpy as np
from zidstats import two_sample_test

rng = np.random.default_rng(0)
X = rng.standard_normal((300, 64))
Y = rng.standard_normal((300, 64)) * 0.85       # same mean, 15% less spread
res = two_sample_test(X, Y)                      # default = ZID omnibus; nothing else to pick
print(res)
# ZID: p=0.00498 (stat=181.4, alpha=0.05) -> reject H0.
# Significant difference driven by SMALLER dispersion (Z_D=-12.4).
res.pvalue, res.statistic, res.reject, res.verdict
res.extra["members"]                       # member p-values: RISE, GPK@median, GPK@0.3med
res.extra["Z_D"], res.extra["Z_D_pval"]    # spread/shape-sensitive arm (from the RISE member)
res.extra["Z_W"], res.extra["Z_W_pval"]    # location-sensitive arm — clean two-axis (from RISE)

Run this first. The default is ZID, a broad-coverage omnibus over three members — RISE, GPK at the median bandwidth, and GPK at a small (0.3×median) bandwidth (GPK@0.3med in extra["members"]). Its p-value comes from a permutation: the members are permuted and flat-Simes-combined, and that combined value is then permuted too, so the final p-value is the permutation p of the combination, not an analytic Simes threshold (exact under exchangeability; omnibus_null="simes" uses the analytic threshold instead). It stays sensitive across location, dispersion, shape, and matched-moment multimodality (the small-bandwidth member catches equal-mean/equal-variance multimodal differences the others miss). It returns one calibrated p-value plus its RISE member's signed Z_W/Z_D arms — location- and spread/shape-sensitive (see Attribution) — and extra["members"] holds the three member p-values.

Attribution (Z_W, Z_D). X and Y are interchangeable — the p-value is the same either way; only the sign of Z_D depends on order (swapping flips it), so put the reference first for a directional read. Z_D is the spread/shape arm, not a variance test: for a pure scale change Z_D<0 reads as Y less dispersed than X, but Z_D also fires on shape, tail, and matched-moment / multimodal differences — it can be significant, in either sign, even when the first two moments of X and Y already match. Z_W is the location arm, but that reading is clean only in moderate-to-high dimension: at low d a spread/shape difference also inflates Z_W, so a large Z_W there does not by itself isolate a mean shift. Both are attribution only; the omnibus p-value does not depend on the split.

How the ZID p-value is built (omnibus_null="perm", the default): under one shared set of n_perm label permutations, (1) each member statistic gets a permutation-rank p, (2) the three are flat-Simes-combined into one value, and (3) that combined value is re-ranked against the same permutations — step (3) is the second permutation, and it's what makes the whole thing exact under exchangeability (no Simes/PRDS assumption). omnibus_null="simes" stops after an analytic Simes threshold on the member p-values (faster; Type-I holds empirically). For a fully analytic GPK, use method="gpk", gpk_null="analytic" (the fast fGPK approximation).

For a quick, fully analytic single-member check — or on very large n — use method="rise": RISE alone is itself a χ²₂ = Z_W²+Z_D² test, robust across geometries, deterministic, no permutation.

Routing — which member when

You almost never need to choose by hand: the default ZID omnibus covers everything. When you do want a specialist (known geometry, max power, or discrete data), route on geometry rather than on the smallest p-value:

your situation	route →	why
anything / unsure (default)	ZID (omnibus)	broad coverage: joint permutation over RISE + GPK@median + GPK@0.3med; location / dispersion / shape / multimodality
quick check / very large n	RISE (method="rise")	fast analytic single-member test (χ²₂), deterministic, no permutation
non-Euclidean / manifold / graph features	RISE	local rank-graph; a global kernel bandwidth smooths manifold density and goes blind to dispersion
plain Euclidean / Gaussian, want max power	GPK	kernel power specialist (Z_W²+Z_D²), strong on location and dispersion in high-d
need attribution (which arm?)	RISE (read Z_W & Z_D)	RISE/GET/GPK give a clean two-axis read
discrete / heavy ties (genotypes, counts, rounded)	GET-discrete (method="get", discrete=True)	validated tied-distance null; the graph/kernel nulls miscalibrate on ties
location said to dominate (Z_D≈0)	a mean test (t / Hotelling)	the verdict flags this; use zidstats when scale/shape may also differ

Non-Euclidean / object data

Euclidean is the default. To use another distance on point inputs, pass metric= — any scipy cdist metric name or a callable f(u, v) -> float; the test then runs DIRECTLY on that dissimilarity (no embedding, no metric assumption):

res = two_sample_test(X, Y, metric="cosine")      # or "cityblock", "braycurtis", ...

If your data is only a precomputed pairwise dissimilarity matrix (graphs, trees, sequences, any custom measure), use two_sample_from_dissimilarity(D, n_x) — it runs the family directly on D (first n_x rows = group 1). The dissimilarity need not be a metric (no triangle inequality).

Member roles

• ZID — the default (omnibus). Run it first: a joint permutation over RISE + GPK@median + GPK@0.3med → one calibrated p-value with the signed Z_W/Z_D. Broad coverage — location, dispersion, shape, matched-moment multimodality. The one call most users ever make.
• RISE — the fast single-member path. method="rise": analytic χ²₂, deterministic, no permutation, manifold-aware, best-calibrated single test. Use for speed or very large n.
• GPK — the Euclidean power specialist. Use when the geometry is plainly Euclidean/Gaussian and you want maximum power on a location or continuous-dispersion difference. Caveat: a single global bandwidth can go blind on manifolds.
• GET — the reference member. The earliest of the family (edge-count); GPK and RISE descend from its within-group idea. A balanced graph alternative.
• GET-discrete — the heavy-ties handler. The validated path for discrete/tied data (genotypes 0/1/2, low counts, rounded values) where continuous nulls break.

Methods

method	type	p-value	sensitive to	tuning (default)
zid	omnibus (default)	two-pass permutation (members permuted → flat-Simes → the combination permuted too) (omnibus_null="perm"; "simes" = analytic Simes over members (RISE analytic χ²₂; GPK members per gpk_null))	location + dispersion + shape + multimodality	sigma=median, small=0.3×; n_perm=200
gpk	kernel	permutation (exact)	location + dispersion (Euclidean)	bandwidth = median
gpk + gpk_null="analytic"	kernel	analytic fGPK (fast approx of gpk; anti-conservative small-n)	location + dispersion	bandwidth = median
rise	graph-induced rank	analytic χ²₂	balance + mode-drop + dependence	k = ⌊N^0.65⌋
get	MST edge-count	analytic χ²₂	scale + dependence	k = ⌊N^0.5⌋

The family tests are distribution-free — validated Python ports of the CRAN packages kerTests / GraphRankTest / gTests.

Attribution arms. RISE and GET are χ²₂ = Z_W²+Z_D² with both arms N(0,1), so they give a clean two-axis read: two orthogonal arms, Z_W (location-sensitive) and Z_D (spread/shape-sensitive), each with an analytic p-value (2·Φ(−|·|)). For the kernel member, GPK = Z_W²+Z_D² too (Song & Chen 2024): its Z_D arm is N(0,1) with an analytic p, while the non-normal Z_W arm (unweighted Z_{W,1.0} ≈ standardized MMD) gets an exact permutation p (extra["Z_W_pval_perm"]). fGPK also exposes the weighted N(0,1) arms ZW1/ZW2.

Note on "kernel" (≠ MMD)

GPK is a kernel test but not MMD: MMD compares the two samples through the cross term and is location-dominated / scale-blind in high dimension; GPK combines the within-group similarities and so detects dispersion collapse (MMD is the Z_W arm alone; GPK adds Z_D).

Cite

Methods: GPK (Song & Chen 2024, Biometrika 111(3):755–770), RISE (Zhou & Chen 2023, COLT/PMLR 195), GET (Chen & Friedman 2017, JASA 112(517):397–409).