HVRT: Hierarchical Variance-Retaining Transformer

Variance-aware sample transformation for tabular data: reduce, expand, or augment. Fits once; operates many times.

Model Family

Five model classes share the same primary API. They differ in how they partition data and which geometric statistic drives the tree.

from hvrt import HVRT, FastHVRT, HART, FastHART, PyramidHART

Algorithm

1. Z-score normalisation

X_z = (X − median) / MAD    (HART / FastHART / PyramidHART)
X_z = (X − mean)   / std    (HVRT / FastHVRT)

2. Synthetic target construction

HVRT / HART — sum of normalised pairwise feature interactions (T-statistic):

T = Σ_{i<j}  normalise(X_z[:,i] ⊙ X_z[:,j])    O(n · d²)

FastHVRT / FastHART — sum of z-scores (S-statistic):

S = Σ_j  X_z[:, j]    O(n · d)

PyramidHART — ℓ₁ cooperation statistic (A-statistic):

A = |S| − ‖z‖₁    bounded, sign-aware, O(n · d)

3. Partitioning

A DecisionTreeRegressor is fitted on the synthetic target. Leaves form variance- homogeneous partitions. Tree depth and leaf size are auto-tuned to dataset size.

4. Per-partition operations

Reduce: Select representatives within each partition using the chosen selection strategy. Budget proportional to partition size (variance_weighted=False) or biased toward high-variance partitions (=True).

Expand: Draw synthetic samples within each partition using the chosen generation strategy. Budget allocation follows the same logic.

Installation

pip install hvrt

git clone https://github.com/jpeaceau/HVRT.git
cd HVRT
pip install -e .

Optional extras:

pip install hvrt[fast]       # Numba-compiled kernels (3–13× speedup on fit/FPS)
pip install hvrt[optimizer]  # Optuna-backed HPO (HVRTOptimizer)
pip install hvrt[benchmarks] # xgboost, matplotlib, pandas for benchmark scripts

Quick Start

HVRT / FastHVRT

from hvrt import HVRT, FastHVRT

# Fit once — reduce and expand from the same model
model = HVRT(random_state=42).fit(X_train, y_train)   # y optional
X_reduced, idx = model.reduce(ratio=0.3, return_indices=True)
X_synthetic    = model.expand(n=50000)
X_augmented    = model.augment(n=15000)

# FastHVRT — O(n·d) target; preferred for expansion
model = FastHVRT(random_state=42).fit(X_train)
X_synthetic = model.expand(n=50000)

PyramidHART with geometry-aware strategies

from hvrt import PyramidHART, geometry_stats

# Fit on features only — sign structure lives in X space
model = PyramidHART(random_state=42).fit(X_train)

# Expand using A-range enforcement (polyhedral rejection sampling)
X_synth = model.expand(n=50000, generation_strategy='a_range_rejection')

# Inspect ℓ₁ geometry per partition
stats = model.geometry_stats()
# [{'partition_id': 0, 'n': 42, 'S_mean': 1.3, 'A_mean': -0.8, ...}, ...]

# Compute geometry statistics on any z-scored array
from hvrt import compute_A
A = compute_A(X_z)   # shape (n,): A-statistic per sample

API Reference

HVRT / FastHVRT

Both expose identical constructor parameters. HVRT uses pairwise interactions (O(d²)); FastHVRT uses z-score sum (O(d)).

from hvrt import HVRT, FastHVRT

model = HVRT(
    n_partitions=None,           # Max tree leaves; auto-tuned if None
    min_samples_leaf=None,       # Min samples per leaf; auto-tuned if None
    max_depth=None,              # Tree max depth; auto-tuned if None
    y_weight=0.0,                # 0.0 = unsupervised; 1.0 = y drives splits
    bandwidth='auto',            # KDE bandwidth: 'auto', float, 'scott', 'silverman'
    auto_tune=True,
    n_jobs=1,                    # Parallelism: -1 = all cores
    tree_splitter='best',        # 'best' or 'random' (10–50× faster fit)
    random_state=42,
    # Pipeline params (see Pipeline section)
    reduce_params=None,
    expand_params=None,
    augment_params=None,
)

HART / FastHART

All constructor parameters identical to HVRT/FastHVRT. Differ in normalisation (median/MAD instead of mean/std) and tree criterion (absolute_error instead of squared_error). Robust to heavy tails and outliers.

from hvrt import HART, FastHART

model = HART(random_state=42).fit(X_train, y_train)
model = FastHART(random_state=42).fit(X_train)

PyramidHART

Extends HART. Uses the ℓ₁ cooperation statistic A = |S| − ‖z‖₁ as the partitioning target. Exposes geometry_stats() for per-partition breakdown of S, Q, T, and A.

from hvrt import PyramidHART

model = PyramidHART(random_state=42).fit(X_train)
stats = model.geometry_stats()   # list of per-partition geometry dicts

HVRTOptimizer

Requires: pip install hvrt[optimizer]

from hvrt import HVRTOptimizer

opt = HVRTOptimizer(
    n_trials=30,             # Optuna trials; use ≥50 in production
    n_jobs=1,                # Parallel trials (-1 = all cores)
    cv=3,                    # Cross-validation folds for the objective
    expansion_ratio=5.0,     # Synthetic-to-real ratio during evaluation
    task='auto',             # 'auto', 'regression', 'classification'
    timeout=None,            # Wall-clock time limit in seconds
    random_state=None,
    verbose=0,               # 0 = silent, 1 = Optuna trial progress
)
opt = opt.fit(X, y)          # y enables TSTR Δ objective; required for classification

Performs TPE-based Bayesian optimisation over n_partitions, min_samples_leaf, y_weight, kernel / bandwidth, and variance_weighted. HVRT defaults are always evaluated as trial 0 (warm start).

Post-fit attributes:

opt = HVRTOptimizer(n_trials=50, n_jobs=4, cv=3, random_state=42).fit(X, y)
print(f'Best TSTR Δ: {opt.best_score_:+.4f}')
X_synth = opt.expand(n=50000)         # y column stripped automatically
X_aug   = opt.augment(n=len(X) * 5)

fit

model.fit(X, y=None)

reduce

X_reduced = model.reduce(
    n=None,                  # Absolute target count
    ratio=None,              # Proportional (e.g. 0.3 = keep 30%)
    method='fps',            # Selection strategy; see Selection Strategies
    variance_weighted=True,  # Oversample high-variance partitions
    return_indices=False,
    n_partitions=None,       # Override tree granularity for this call only
)

expand

X_synth = model.expand(
    n=10000,
    variance_weighted=False,      # True = oversample tails
    bandwidth=None,               # Override instance bandwidth
    adaptive_bandwidth=False,     # Scale bandwidth with local expansion ratio
    generation_strategy=None,     # See Generation Strategies
    return_novelty_stats=False,
    n_partitions=None,
)

augment

X_aug = model.augment(n=15000, variance_weighted=False)
# n must exceed len(X); returns original X concatenated with synthetic samples

Utility methods

partitions = model.get_partitions()
# [{'id': 5, 'size': 120, 'mean_abs_z': 0.84, 'variance': 1.2}, ...]

novelty = model.compute_novelty(X_new)   # min z-space distance per point

params = HVRT.recommend_params(X)        # {'n_partitions': 180, ...}

# PyramidHART only
stats = model.geometry_stats()
# [{'partition_id': 0, 'n': 42, 'S_mean': 1.3, 'Q_mean': 0.5, 'T_mean': 1.1,
#   'A_mean': -0.8, 'mst_mean': 0.4, 'A_q05': -2.1, 'A_q95': 0.3}, ...]

sklearn Pipeline

Operation parameters are declared at construction time via ReduceParams, ExpandParams, or AugmentParams. The tree is fitted once during fit(); transform() calls the corresponding operation.

from hvrt import HVRT, FastHVRT, ReduceParams, ExpandParams, AugmentParams
from sklearn.pipeline import Pipeline

# Reduce
pipe = Pipeline([('hvrt', HVRT(reduce_params=ReduceParams(ratio=0.3)))])
X_red = pipe.fit_transform(X, y)

# Expand
pipe = Pipeline([('hvrt', FastHVRT(expand_params=ExpandParams(n=50000)))])
X_synth = pipe.fit_transform(X)

# Augment
pipe = Pipeline([('hvrt', HVRT(augment_params=AugmentParams(n=15000)))])
X_aug = pipe.fit_transform(X)

Import from hvrt.pipeline to make intent explicit:

from hvrt.pipeline import HVRT, ReduceParams

ReduceParams

ReduceParams(
    n=None,
    ratio=None,              # e.g. 0.3
    method='fps',
    variance_weighted=True,
    return_indices=False,
    n_partitions=None,
)

ExpandParams

ExpandParams(
    n=50000,                 # required
    variance_weighted=False,
    bandwidth=None,
    adaptive_bandwidth=False,
    generation_strategy=None,
    return_novelty_stats=False,
    n_partitions=None,
)

AugmentParams

AugmentParams(
    n=15000,                 # required; must exceed len(X)
    variance_weighted=False,
    n_partitions=None,
)

Generation Strategies

Seven built-in strategies: four general-purpose and three PyramidHART-specific.

from hvrt import FastHVRT, epanechnikov, univariate_kde_copula

model = FastHVRT(random_state=42).fit(X)

# By name (preferred)
X_synth = model.expand(n=10000, generation_strategy='epanechnikov')

# By reference
X_synth = model.expand(n=10000, generation_strategy=univariate_kde_copula)

PyramidHART-specific strategies

These strategies encode assumptions about the ℓ₁ polyhedral geometry of PyramidHART: the cooperation statistic A = |S| − ‖z‖₁ partitions feature space into sign-coherent cones, and these strategies are designed to preserve or restore that structure.

Important: Fit on X only (no y column stacked). Stacking y introduces a sign dimension unrelated to the geometric construction.

from hvrt import PyramidHART

model = PyramidHART(random_state=42).fit(X_train)  # X only

# A-range enforcement: reject samples outside training A-quantile range
X_synth = model.expand(n=50000, generation_strategy='a_range_rejection')

# Sign-preserving: Epanechnikov on magnitudes, original signs restored
X_synth = model.expand(n=50000, generation_strategy='sign_preserving_epanechnikov')

# MST-matching: bootstrap minority-sign group to match training MST
X_synth = model.expand(n=50000, generation_strategy='minority_sign_resampler')

All three strategies produce the same results as standard Epanechnikov at the default auto-tuned partition granularity (n≈500, ~18–20 leaves). Their geometric advantages emerge at larger n and finer partitions (n_partitions≥50) where sign structure in A is more pronounced.

Custom strategy

from hvrt import StatefulGenerationStrategy, PartitionContext
import numpy as np

class MyStrategy:
    def prepare(self, X_z, partition_ids, unique_partitions):
        # precompute partition metadata once; return a PartitionContext subclass
        ...
        return PartitionContext(X_z=X_z, ...)

    def generate(self, context, budgets, random_state):
        ...
        return X_synthetic   # shape (sum(budgets), n_features), z-score space

X_synth = model.expand(n=10000, generation_strategy=MyStrategy())

from hvrt import BUILTIN_GENERATION_STRATEGIES
list(BUILTIN_GENERATION_STRATEGIES)
# ['multivariate_kde', 'univariate_kde_copula', 'bootstrap_noise', 'epanechnikov',
#  'a_range_rejection', 'sign_preserving_epanechnikov', 'minority_sign_resampler']

Selection Strategies

from hvrt import HVRT

model = HVRT(random_state=42).fit(X, y)

# By name (preferred)
X_red = model.reduce(ratio=0.2, method='fps')             # default
X_red = model.reduce(ratio=0.2, method='medoid_fps')
X_red = model.reduce(ratio=0.2, method='variance_ordered')
X_red = model.reduce(ratio=0.2, method='stratified')

Custom strategy

from hvrt import StatefulSelectionStrategy, SelectionContext
import numpy as np

class MySelector:
    def prepare(self, X_z, partition_ids, unique_partitions):
        from hvrt.reduction_strategies import _build_selection_context
        return _build_selection_context(X_z, partition_ids, unique_partitions)

    def select(self, context, budgets, random_state, n_jobs=1):
        ...
        return selected_indices   # global indices into X_z

X_red = model.reduce(ratio=0.2, method=MySelector())

Memory-conscious large-data workflow:

model = HVRT(n_jobs=-1).fit(X_large)          # n_jobs forwarded to select()
X_red = model.reduce(ratio=0.1, method='fps') # parallel FPS, O(partition size) memory per worker

Cooperative Geometry

The _geometry.py module provides standalone functions for computing ℓ₁ cooperation statistics. These are useful for model selection, diagnostics, and custom analysis.

Definitions

Usage

from hvrt import compute_A, geometry_stats

# Compute A-statistic on z-scored feature matrix
import numpy as np
X_z = (X - X.mean(0)) / X.std(0)
A = compute_A(X_z)   # shape (n,); A ∈ [−d/2, 0]

# Full geometry stats (S, Q, T, A) per sample
from hvrt._geometry import compute_S, compute_Q, compute_T, minority_sign_total
S = compute_S(X_z)   # shape (n,)
Q = compute_Q(X_z)   # shape (n,)
T = compute_T(X_z)   # S² − Q, shape (n,)
mst = minority_sign_total(X_z)   # shape (n,)

# Per-partition breakdown from a fitted PyramidHART model
model = PyramidHART().fit(X_train)
stats = model.geometry_stats()
# [{'partition_id': 0, 'n': 42, 'S_mean': 1.3, 'A_mean': -0.8, ...}, ...]

When to use each model

Recommendations

bandwidth='auto' — the default

bandwidth='auto' requires no tuning for most datasets. At each expand() call it inspects the fitted partition structure and picks the kernel most likely to produce high-quality synthetic data.

How it decides:

Why not Scott's rule: Scott's rule assumes iid Gaussian data. HVRT partitions are locally homogeneous but non-Gaussian (mean |skewness| 0.49–1.37 across benchmark datasets). The decision tree already captures the primary variance structure, so the residual within-partition variance is narrower than Scott's formula assumes, causing systematic over-smoothing. Scott's rule won 0 of 18 benchmark conditions.

When to override:

• Heterogeneous / high-skew classification (mean |skew| ≳ 0.8): use generation_strategy='epanechnikov' directly.
• Small dataset, coarse partitions, regression: use bandwidth=0.1 or bandwidth=0.3.

Model selection guidance

Hyperparameter optimisation (HPO)

Dataset heterogeneity is the primary driver of sensitivity to HVRT's parameters. A well-behaved near-Gaussian dataset produces good synthetic data at defaults. A dataset with distinct clusters or regime-switching needs finer partitions.

Parameter search space:

When HPO is worth running:

• TSTR Δ is significantly negative (below −0.05)
• Dataset has known sub-populations, clusters, or regime changes
• Generating at high ratio (10×+)

from hvrt import HVRTOptimizer

opt = HVRTOptimizer(n_trials=50, n_jobs=4, cv=3, random_state=42).fit(X, y)
print(f'Best TSTR Δ: {opt.best_score_:+.4f}')
X_synth = opt.expand(n=50000)
X_aug   = opt.augment(n=len(X) * 5)

Benchmarks

Sample reduction

Metric: GBM ROC-AUC on reduced training set as % of full-training-set AUC. n=3 000 train / 2 000 test, seed=42.

HVRT-fps: method='fps', variance_weighted=True. HVRT-yw: same + y_weight=0.3.

Reproduce: python benchmarks/reduction_denoising_benchmark.py

Synthetic data expansion

Metrics: discriminator accuracy (target ≈ 50%), marginal fidelity, tail preservation (target = 1.0), Privacy DCR, TSTR Δ. bandwidth='auto', max_n=500 training samples, expansion ratio 1×. Mean across continuous benchmark datasets (fraud, housing, multimodal).

† PyramidHART-ARejection uses X-only fit + proxy y — correct evaluation for geometry-aware strategies. ‡ CTGAN/TVAE run locally (--deep-learning). Poor Disc. Err reflects small n=400 training set. § Published numbers only — no local runner. * CTGAN/TVAE TRTR is 0.769 (housing + multimodal only; fraud not evaluated). Disc. Err = |discriminator accuracy − 50%|. Lower = more indistinguishable from real.

Reproduce: python benchmarks/run_benchmarks.py --tasks expand --deep-learning

Privacy evaluation

The benchmark suite computes two privacy metrics for every expansion run.

Distance-to-Closest-Record (DCR)

DCR = median(min_dist(synthetic_i → real))
    / median(min_dist(real_i → real excluding itself))

HVRT (DCR ≈ 0.45) is 3× safer than Bootstrap + Noise (DCR ≈ 0.16) and 1.5× safer than SMOTE (DCR ≈ 0.30) on continuous data.

Privacy–Fidelity Decision Matrix

Reproduce: python benchmarks/dcr_privacy_benchmark.py

Benchmarking Scripts

python benchmarks/run_benchmarks.py
python benchmarks/run_benchmarks.py --tasks reduce --datasets adult housing
python benchmarks/run_benchmarks.py --tasks expand
python benchmarks/pyramid_hart_benchmark.py          # PyramidHART vs HART/HVRT family
python benchmarks/pyramid_hart_benchmark.py --quick  # 2 datasets, fast check
python benchmarks/strategy_speedup_benchmark.py      # vectorization speedup
python benchmarks/speed_benchmark.py                 # serial vs parallel wall-clock
python benchmarks/reduction_denoising_benchmark.py
python benchmarks/hpo_benchmark.py                  # requires: pip install hvrt[optimizer]
python benchmarks/dcr_privacy_benchmark.py           # privacy–fidelity sweep

Testing

pytest
pytest --cov=hvrt --cov-report=term-missing

Citation

@software{hvrt2026,
  author = {Peace, Jake},
  title  = {HVRT: Hierarchical Variance-Retaining Transformer},
  year   = {2026},
  url    = {https://github.com/jpeaceau/HVRT}
}

License

GNU Affero General Public License v3 or later (AGPL-3.0-or-later) — see LICENSE.

Acknowledgments

Development assisted by Claude (Anthropic).