hedged-rf 1.0.0

Hedged Random Forest — optimized non-equal tree weights for time-series forecasting

hedged-rf

Hedged Random Forest (HRF) — optimized non-equal tree weights for time-series forecasting.

---

Overview

hedged-rf is a Python implementation of the Hedged Random Forest methodology introduced in:

Beck, E., & Wolf, M. (2025). Forecasting Inflation with the Hedged Random Forest.
SNB Working Papers 07/2025, Swiss National Bank.

The standard Random Forest averages all tree predictions with equal weights. The HRF instead solves a constrained optimization problem to find weights that minimize the mean-squared forecast error — and crucially, allows negative weights, which has been shown to improve accuracy in volatile economic environments.

Key features

• EWMA estimation — gives more weight to recent observations, suitable for non-stationary economic time series
• Linear shrinkage — regularizes the mean vector and covariance matrix toward structured targets (CVC), ensuring well-conditioned estimates even with hundreds of trees
• Gross-exposure constraint (κ) — controls how extreme the weights can be, preventing overfitting
• Numba-accelerated — parallelized HAC variance estimation for large forests (500–1000 trees)
• CVXPY / OSQP solver — reliable convex optimization backend

---

Installation

pip install hedged-rf

Optional extras

# Development tools (linting, testing)
pip install "hedged-rf[dev]"

# Example notebooks
pip install "hedged-rf[examples]"

# Documentation builder
pip install "hedged-rf[docs]"

Requirements

Package	Version
Python	≥ 3.9
numpy	≥ 1.23
scipy	≥ 1.9
cvxpy	≥ 1.3
numba	≥ 0.57
scikit-learn	≥ 1.2

---

Quick start

import numpy as np
from sklearn.ensemble import RandomForestRegressor
from hedged_rf import fit_hrf, extract_tree_predictions

# 1. Train a standard Random Forest
rf = RandomForestRegressor(n_estimators=500, random_state=42)
rf.fit(X_train, y_train)

# 2. Build the residual matrix  R  (T × n_trees)
R = extract_tree_predictions(rf, X_train, y_train)

# 3. Estimate optimal HRF weights
w, mu, Sigma = fit_hrf(
    R,
    lambda_param=0.15,   # EWMA decay (0.15 recommended for monthly data)
    H=6,                 # bandwidth for autocovariance estimation
    kappa=2.0,           # gross-exposure constraint (allows moderate negative weights)
    verbose=True,
)

# 4. Generate out-of-sample forecasts
tree_preds = np.column_stack([tree.predict(X_test) for tree in rf.estimators_])
y_pred_hrf = tree_preds @ w

---

API reference

fit_hrf(R, lambda_param, H, kappa, verbose, use_fast)

Main entry point. Estimates optimal weights for the Hedged Random Forest.

Parameter	Type	Default	Description
R	np.ndarray (T × N)	—	Residual matrix from training data
lambda_param	float	0.15	EWMA decay parameter ∈ (0, 1)
H	int	6	Bandwidth for HAC autocovariance estimation
kappa	float	2.0	L₁ gross-exposure constraint
verbose	bool	False	Print estimation progress
use_fast	bool	True	Enable Numba parallel acceleration

Returns (w, mu, Sigma):

• w — optimal weight vector (N,)
• mu — estimated mean vector (N,)
• Sigma — estimated covariance matrix (N × N)

---

extract_tree_predictions(rf, X, y)

Builds the (T × N) residual matrix from a fitted scikit-learn RandomForestRegressor.

R = extract_tree_predictions(rf, X_train, y_train)
# R.shape → (n_samples, n_estimators)

---

estimate_mu_complete(R, lambda_param, H, verbose, use_fast)

Stand-alone EWMA + linear-shrinkage estimator for the mean vector μ.

---

estimate_sigma_complete(R, lambda_param, H, verbose, use_fast)

Stand-alone EWMA + linear-shrinkage estimator for the covariance matrix Σ.

---

solve_hrf_optimization(mu, Sigma, kappa, verbose)

Stand-alone convex solver. Solves:

min_w  (w'μ)² + w'Σw
s.t.   w'1 = 1
       ‖w‖₁ ≤ κ

---

Methodology

Why non-equal weights?

A Random Forest minimizes variance by averaging independent trees, but it ignores the bias of individual trees and their error correlations. The HRF weights minimize the full MSE decomposition:

MSE(f̂_w) = (w'μ)² + w'Σw

where μ is the vector of tree bias terms and Σ is the tree-error covariance matrix.

Why negative weights?

When some trees are systematically biased in the same direction, a negative weight on those trees can cancel out that bias, reducing the overall forecast error. The gross-exposure constraint κ keeps negative positions bounded.

The EWMA + shrinkage pipeline

For time-series data (non-i.i.d.), equal-weighted sample estimators of μ and Σ are suboptimal. The HRF instead uses:

1. EWMA estimation — exponentially decaying weights give more importance to recent observations, adapting to structural breaks and ARCH/GARCH effects
2. Linear shrinkage to CVC target — regularizes Σ̂ toward a constant-variance-covariance matrix, using a data-driven shrinkage intensity α = ν/(ν+γ) derived from HAC variance estimates

Choosing lambda_param (λ)

Data frequency	Recommended λ
Daily	0.06
Weekly	0.10
Monthly	0.15 (default)
Quarterly	0.25

Larger λ → faster decay → more weight on recent observations.

Choosing kappa (κ)

Value	Effect
1.0	All weights non-negative (like standard weighted RF)
2.0	Recommended default — moderate negative weights allowed
> 2.0	More aggressive hedging; may overfit on small samples

---

Empirical results

From Beck & Wolf (2025), using US and Swiss inflation data (1990–2023):

Metric	Typical improvement
RMSE vs standard RF	~4% reduction (up to 7% for core inflation)
MAE vs standard RF	~5% reduction (up to 8% for core inflation)

The HRF outperforms the standard RF consistently across all 6 inflation measures and all 12 forecast horizons tested.

---

Examples

See the examples/ directory for notebooks covering:

• basic_usage.ipynb — end-to-end walkthrough with synthetic data
• inflation_forecasting.ipynb — replication of the Beck & Wolf (2025) results using FRED-MD data
• hyperparameter_sensitivity.ipynb — sensitivity analysis for λ and κ

---

Citation

If you use hedged-rf in your research, please cite the original paper:

@techreport{beck2025hrf,
  title   = {Forecasting Inflation with the Hedged Random Forest},
  author  = {Beck, Elliot and Wolf, Michael},
  year    = {2025},
  institution = {Swiss National Bank},
  type    = {SNB Working Papers},
  number  = {07/2025}
}

---

Contributing

Contributions are welcome! Please open an issue or pull request on GitHub.

git clone https://github.com/Ezequiel025/hedged-rf.git
cd hedged-rf
pip install -e ".[dev]"
pre-commit install
pytest

---

License

This project is licensed under the MIT License — see LICENSE for details.