RiskReplay

RiskReplay is an outcome-aware evaluation engine for financial AI systems operating under delayed and selective labels. It is built for teams that need more than a drift alert: they need to know whether a metric is trustworthy, what it means financially, and which cases a human reviewer should inspect first.

RiskReplay produces evidence and prioritization support. It is not an AML decision engine, a credit-decision service, or a compliance certification tool.

What it does

Install

pip install riskreplay

For local development:

pip install -e ".[dev]"

Minimal evaluation

from riskreplay import ColumnSpec, ObservationPolicy, ReviewPolicy, evaluate

report = evaluate(
    events="/secure/path/aml_alerts.parquet",
    reference="/secure/path/reference_window.parquet",
    columns=ColumnSpec(
        id="alert_id", score="aml_score", label="confirmed_suspicious",
        label_status="label_status", exposure="amount_usd",
        uncertainty="model_uncertainty", ood_score="network_ood_score",
        segment="channel", model_version="model_version",
    ),
    observation=ObservationPolicy(maturity_days=45),
    review=ReviewPolicy(budget=500),
    slices=["channel", "risk_band"],
)
report.write_evidence_bundle("artifacts/run-2026-08-23")

The important constraint

Labels are not always a random sample. If only some alerts are reviewed, a score calculated from observed labels can be biased. RiskReplay reports label_coverage, pending_rate, and identifiability alongside model metrics. If it has no defensible observation information, it does not label a selection-corrected metric as reliable.

Privacy

RiskReplay runs locally. The evidence bundle stores aggregate metrics, configuration, timestamps, and content fingerprints by default; it does not export raw customer records unless an application explicitly writes the review queue.

Public assets

The public Hugging Face catalog at adnanallemon/riskreplay-public-assets holds only publication-safe schema and reproducibility material. It intentionally does not contain production financial records or simulated customer data.

Project status

0.1.0 is the first Python MVP. The Rust crate in crates/riskreplay-core contains deterministic primitives that can be exposed through PyO3 in a later performance release without changing the public Python contract.

License

Apache-2.0.