Skip to main content

Model Risk Validator for Model Risk Governance

Project description

mrv-lib: Model Risk Validator

CI PyPI Python

Your regime model might be producing different risk labels depending on which features you feed it — and your current validation doesn't catch this. Most regime models (GMM, HMM) are validated on in-sample fit, but nobody checks whether the labels survive a change in input representation. If they don't, downstream risk decisions are sitting on an arbitrary modelling choice. mrv-lib tests for this.

mrv tests whether your market regime model produces stable, reliable labels — or whether they silently depend on undisclosed modelling choices like feature selection, temporal resolution, or rolling window parameters. Built for SR 11-7 / Basel IV model risk governance.

What it does

Test Question Status
Representation Invariance Do regime labels change when you use different risk factors? v0.1.0
Resolution Invariance Do labels agree across 5m / 15m / 1h / 1d frequencies? v0.2.0
RSS — Regime Stability Score A single score combining rep + res into actionable governance signal v0.3.0

v0.2.0 also includes: business impact function (impact_fn), continuous monitoring with alerts, disagreement attribution (LOO / frequency-pair / temporal), SR 11-7 compliant report with auto-generated findings, and a findings engine with severity classification.

Install

pip install mrv-lib

# Optional: IB data download
pip install mrv-lib[ib]

# Optional: regime models (GMM/HMM)
pip install mrv-lib[validator]

# Everything
pip install mrv-lib[all]

Quick start

Notebooks: See examples/paper1_representation_invariance.ipynb and examples/paper2_resolution_invariance.ipynb for step-by-step walkthroughs.

# 1. Download data (requires IB Gateway running)
python run.py download config.yaml

# 2. Run representation invariance test + generate PDF
python run.py run config.yaml rep

# 3. Run resolution invariance test + generate PDF
python run.py run config.yaml res

# 4. Regenerate PDF from existing results
python run.py report

Or from Python:

from mrv.pipeline import run, download, validate, report

download("config.yaml")             # fetch data from IB
run("config.yaml", "rep")           # validate + PDF
run("config.yaml", "res")           # validate + PDF

# Step by step (full control)
from mrv.pipeline import load_data, compute_factors, fit_labels, validate, report
from mrv.utils.config import load

cfg = load("config.yaml")
prices = load_data(cfg, "rep")
# ... user can replace any step

Configuration

All settings in one config.yaml:

download:
  data_dir: data
  symbols: [SPY, USDJPY, CL=F, IEF, GLD]
  freq: [5m, 15m, 1h, 1d]
  start: "2026-01-01"
  ib:
    host: 127.0.0.1
    port: 4002

factors:
  vol_window: 20
  drawdown_window: 60
  tail_window: 60
  tail_alpha: 0.05

validator:
  rep:
    assets:
      SPY: data/SPY_1d.csv
    model: gmm
    n_states: 3
    factors:
      - [vol, drawdown, maxdd, var, cvar]
      - [vol, drawdown, var, cvar]
      - [real_skew, vol_stab, var, cvar]

  res:
    assets:
      SPY: [data/SPY_5m.csv]
    model: gmm
    n_states: 2
    episode: 2026_iran

Custom factors and models

from mrv.data.factors import register_factor

def momentum(returns, price, windows):
    return price.pct_change(windows.get("mom_window", 20)).rename("momentum")

register_factor("momentum", momentum)
from mrv.models import register_model

def my_kmeans(features, n_states, **kwargs):
    from sklearn.cluster import KMeans
    return KMeans(n_clusters=n_states).fit_predict(features.values)

register_model("kmeans", my_kmeans)

Project layout

mrv-lib/
├── config.yaml              # Configuration
├── run.py                   # CLI entry point
├── templates/
│   ├── template.tex         # Academic report template
│   └── sr11_7_template.tex  # SR 11-7 regulatory report template
├── examples/
│   ├── paper1_representation_invariance.ipynb
│   └── paper2_resolution_invariance.ipynb
├── src/mrv/
│   ├── pipeline.py          # data -> factors -> model -> validate -> report
│   ├── data/
│   │   ├── reader.py        # Load, validate, resample OHLCV
│   │   ├── factors.py       # Factor registry + 7 built-in risk factors
│   │   └── normalize.py     # Rolling z-score, min-max
│   ├── models/
│   │   ├── __init__.py      # Model registry + fit()
│   │   ├── gmm.py           # Gaussian Mixture Model
│   │   └── hmm.py           # Hidden Markov Model
│   ├── validator/
│   │   ├── base.py          # BaseValidator (subclass for custom tests)
│   │   ├── rep.py           # Representation Invariance test (Paper 1)
│   │   ├── res.py           # Resolution Invariance test (Paper 2)
│   │   ├── metrics.py       # ARI, AMI, NMI, Spearman, VI
│   │   ├── attribution.py   # LOO, frequency-pair, temporal hotspots
│   │   ├── findings.py      # SR 11-7 findings engine
│   │   ├── monitor.py       # Continuous monitoring + alerts
│   │   └── report.py        # JSON -> LaTeX -> PDF
│   └── utils/
│       ├── config.py        # YAML config loading
│       ├── download.py      # IB data download
│       └── log.py           # Logging setup
├── reports/                  # Output (gitignored)
└── tests/                    # 123 tests

Output

Each run creates a timestamped directory under reports/:

  • result.json -- Complete data (reusable for report regeneration)
  • report.pdf -- Professional report with cover page, dashboard, heatmaps, and remediation plan
  • summary.txt -- Plain text quick view
  • {asset}_ari_heatmap.png -- ARI heatmap per asset
  • {asset}_timeline.png -- Regime timeline (res validator)
  • pipeline_summary.csv -- Summary metrics per asset

Research

Based on the following PhD research:

  • Zheng, Low & Wang (2026). Regime Labels Are Not Representation-Invariant. Submitted.
  • Zheng, Low & Wang (2026). Regime Labels Are Not Resolution-Invariant. Submitted to Finance Research Letters.

License

MIT. See LICENSE.

Maintainers

ModelGuard Lab -- Author: Kai Zheng.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mrv_lib-0.2.1.tar.gz (55.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

mrv_lib-0.2.1-py3-none-any.whl (59.0 kB view details)

Uploaded Python 3

File details

Details for the file mrv_lib-0.2.1.tar.gz.

File metadata

  • Download URL: mrv_lib-0.2.1.tar.gz
  • Upload date:
  • Size: 55.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for mrv_lib-0.2.1.tar.gz
Algorithm Hash digest
SHA256 7406279d9a51dc04c4b8f97538e198db563674c968379f2b2e9d0097425711c3
MD5 419e6373f451edf8394c6a7066ab601c
BLAKE2b-256 f94efef3236a38420da329d9581caed606b8eb7916da220ca68e746f2c58e51f

See more details on using hashes here.

Provenance

The following attestation bundles were made for mrv_lib-0.2.1.tar.gz:

Publisher: publish.yml on modelguard-lab/mrv-lib

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mrv_lib-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: mrv_lib-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 59.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for mrv_lib-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 e475ef3db609fe9fde114f227d2e816e1858988b006ca245de26347289e80429
MD5 d28ee2bdbef5108f46845ab0cf5df30a
BLAKE2b-256 ef7d37f7cc46d032a8a6df7a0fecd6ef4cf4eda7e0035db251610b485911feba

See more details on using hashes here.

Provenance

The following attestation bundles were made for mrv_lib-0.2.1-py3-none-any.whl:

Publisher: publish.yml on modelguard-lab/mrv-lib

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page