insurance-deploy 0.1.0

Champion/challenger pricing framework for UK insurance — model registry, quote routing, ENBP audit logging, and statistical promotion tests

insurance-deploy

Champion/challenger pricing framework for UK insurance — model registry, quote routing, ENBP audit logging, and statistical promotion tests.

---

The problem

You've built a better pricing model. CatBoost instead of GLM, or an updated GLM with two years more data and a rebuilt rating factor for NCB. The model validates well in holdout. Your actuarial team wants to deploy it.

The problem is everything after model training.

How do you run the challenger alongside the champion without disrupting live pricing? How do you log which model priced each quote — per-quote, per-policy, permanently — so you can run the FCA-required ENBP audit? How do you know when you have enough data to make a statistically credible promotion decision, rather than guessing after three months on a sample too small to tell you anything?

Every UK pricing team faces this. Most solve it with ad-hoc scripts, spreadsheet logs, and informal sign-off. This library provides the infrastructure.

---

Regulatory context

ICOBS 6B.2.51R (the ENBP rules, effective January 2022) requires firms to maintain written records demonstrating that renewal prices do not exceed the Equivalent New Business Price for identical risk profiles.

FCA multi-firm review (2023): 83% of firms were non-compliant with record-keeping requirements. Most lacked records granular enough for the SMF holder to sign the annual attestation.

When a pricing model changes mid-year, you must be able to demonstrate which model priced each renewal and that the model change did not introduce tenure discrimination. That requires a per-quote model version log. This library is that log.

FCA Consumer Duty (PRIN 2A) creates a risk for live A/B pricing. Charging two customers of identical profile differently simultaneously could be challenged as inconsistent with fair value obligations. Shadow mode (the default) eliminates this risk — challenger scores in parallel but the customer always sees the champion price.

---

What this library does

Five modules:

Module	Contents
insurance_deploy.registry	ModelRegistry — append-only version-tagged model store with hash verification
insurance_deploy.experiment	Experiment — deterministic hash-based routing, shadow and live modes
insurance_deploy.logger	QuoteLogger — append-only SQLite audit log with ENBP compliance flagging
insurance_deploy.kpi	KPITracker — hit rate, GWP, loss ratio, frequency, power analysis
insurance_deploy.comparison	ModelComparison — bootstrap LR test, z-test on hit rate, Poisson frequency test
insurance_deploy.audit	ENBPAuditReport — ICOBS 6B.2.51R compliance report in Markdown

---

Install

pip install insurance-deploy

Dependencies: NumPy, SciPy, Pandas, joblib.

---

Quick start

Register models

from insurance_deploy import ModelRegistry

registry = ModelRegistry("./registry")

# Register current champion
champion_mv = registry.register(
    champion_model,           # any sklearn-compatible object with .predict()
    name="motor",
    version="2.0",
    metadata={
        "training_date": "2024-01-01",
        "features": ["age", "ncd", "postcode_band"],
        "holdout_gini": 0.42,
    }
)

# Register challenger
challenger_mv = registry.register(
    challenger_model,
    name="motor",
    version="3.0",
    metadata={
        "training_date": "2024-07-01",
        "features": ["age", "ncd", "postcode_band", "vehicle_value"],
        "holdout_gini": 0.45,
    }
)

Set up the experiment

from insurance_deploy import Experiment

exp = Experiment(
    name="motor_v3_vs_v2",
    champion=champion_mv,
    challenger=challenger_mv,
    challenger_pct=0.10,  # 10% of policies to challenger
    mode="shadow",        # Default. Challenger scores but does not price.
)

Route quotes and log

from insurance_deploy import QuoteLogger

logger = QuoteLogger("./quotes.db")

# In your quote handler:
def handle_quote(policy_id, inputs, renewal_flag=False, enbp=None):
    arm = exp.route(policy_id)

    # Champion always prices in shadow mode
    champion_price = champion_mv.predict([inputs])[0]

    # Challenger scores in shadow — output logged, not shown to customer
    challenger_price = challenger_mv.predict([inputs])[0]

    # Log champion quote (the one the customer sees)
    logger.log_quote(
        policy_id=policy_id,
        experiment_name=exp.name,
        arm=arm,
        model_version=champion_mv.version_id,  # champion always prices
        quoted_price=champion_price,
        enbp=enbp,           # Provide for renewals — you calculate this, not us
        renewal_flag=renewal_flag,
    )

    return champion_price

# When policy binds:
logger.log_bind("POL-12345", bound_price=425.0)

# When claim is reported (log at each development stage):
from datetime import date
logger.log_claim("POL-12345", claim_date=date(2024, 8, 1),
                 claim_amount=1200.0, development_month=3)
# Update at 12 months:
logger.log_claim("POL-12345", claim_date=date(2024, 8, 1),
                 claim_amount=1450.0, development_month=12)

Track KPIs

from insurance_deploy import KPITracker

tracker = KPITracker(logger)

# Immediately available
print(tracker.hit_rate("motor_v3_vs_v2"))
# {'champion': {'quoted': 900, 'bound': 270, 'hit_rate': 0.30},
#  'challenger': {'quoted': 100, 'bound': 28, 'hit_rate': 0.28}}

print(tracker.gwp("motor_v3_vs_v2"))
# {'champion': {'bound_policies': 270, 'total_gwp': 108000.0, 'mean_gwp': 400.0},
#  'challenger': {'bound_policies': 28, 'total_gwp': 11480.0, 'mean_gwp': 410.0}}

# After 12 months development
lr = tracker.loss_ratio("motor_v3_vs_v2", development_months=12)
print(lr)
# {'champion': {'loss_ratio': 0.64, 'policy_count': 270, ...},
#  'challenger': {'loss_ratio': 0.61, 'policy_count': 28, ...}}

# Power analysis: how long until we can decide?
pa = tracker.power_analysis("motor_v3_vs_v2", target_delta_lr=0.03)
print(f"Months to LR significance (incl. 12m development): "
      f"{pa['lr_total_months_with_development']:.0f}")
# Months to LR significance (incl. 12m development): 28

Statistical comparison

from insurance_deploy import ModelComparison

comp = ModelComparison(tracker)

# Bootstrap loss ratio test (requires 12m+ development)
result = comp.bootstrap_lr_test("motor_v3_vs_v2", n_bootstrap=10_000, seed=42)
print(result.summary())
# Test: bootstrap_lr_test | Experiment: motor_v3_vs_v2
# Champion estimate: 0.6402 (n=270)
# Challenger estimate: 0.6118 (n=28)
# Difference (challenger - champion): -0.0284
# 95% CI: [-0.0751, 0.0183]
# p-value: 0.2341
#
# Conclusion: INSUFFICIENT_EVIDENCE
# Recommendation: No statistically significant difference detected in loss_ratio
# (p=0.234 >= 0.05). Continue experiment. Consider running power_analysis()
# to estimate time to significance.

# Hit rate test (available earlier)
hr_result = comp.hit_rate_test("motor_v3_vs_v2")

ENBP audit report

from insurance_deploy import ENBPAuditReport

reporter = ENBPAuditReport(logger)
md = reporter.generate(
    "motor_v3_vs_v2",
    period_start="2024-01-01",
    period_end="2024-12-31",
    firm_name="Acme Insurance Ltd",
    smf_holder="Jane Smith",
)
print(md)  # Markdown: paste into attestation pack or Databricks notebook

---

Shadow mode vs live mode

Shadow mode is the default and is right for most use cases.

Shadow mode: champion prices every quote. Challenger runs on identical inputs, output is logged but not shown to the customer. Zero fair value regulatory risk. Model quality comparison is clean — no adverse selection confound.

Live mode: challenger prices its routed fraction of policies (10% by default). Enables market response testing (does challenger's different pricing affect conversion?). Raises FCA Consumer Duty fair value questions. Also introduces adverse selection bias: if challenger prices differently, the bound cohorts will have different risk profiles, making loss ratio comparison harder to interpret.

# Live mode — get legal sign-off first
import warnings
with warnings.catch_warnings():
    warnings.simplefilter("ignore")  # suppress the FCA warning if you've taken legal advice
    exp = Experiment(
        name="live_test",
        champion=champion_mv,
        challenger=challenger_mv,
        mode="live",
    )

The library is opinionated here. The warning is intentional. Suppress it when you've done the legal groundwork, not before.

---

Routing determinism

Routing uses SHA-256(policy_id + experiment_name), last 8 hex characters as integer, modulo 100. If result < challenger_pct * 100, route to challenger.

This is deterministic and stateless. Given a policy_id and experiment name, the routing decision is always the same and can be recomputed at any point independently. No database of assignments required. Any assignment can be verified from first principles.

Assignment is by policy, not by quote. A policy routed to challenger on first quote will always be routed to challenger within this experiment. This is required for ENBP audit integrity.

---

Why loss ratio significance takes 18 months

At 10% challenger split with 3,000 bound policies/month total:

• Challenger receives ~300 policies/month
• Hit rate significance (2pp delta, 80% power): ~5 months
• Claim frequency significance (0.5pp delta): ~10 months
• Developed loss ratio significance (3pp delta, 12-month development): ~17 months from first quote, total ~29 months from experiment start

This is not a limitation of the library. It is physics. LR has a 12-36 month reward tail. Any framework claiming to optimise on LR signal faster than this using bandits or similar methods is either using a proxy metric (hit rate) or lying.

The power_analysis() method makes this timeline explicit. Run it before starting an experiment so your stakeholders have realistic expectations.

---

Radar wrapper pattern

Most UK personal lines insurers deploy rates via WTW Radar Live. The library integrates as a governance layer around Radar:

import requests
from insurance_deploy import Experiment, QuoteLogger

def get_quote(policy_id, risk_dict, renewal_flag=False, enbp=None):
    # Champion = Radar Live (existing production system)
    radar_response = requests.post(RADAR_LIVE_URL, json=risk_dict)
    champion_price = radar_response.json()["premium"]

    # Challenger = Python model (your new model)
    arm = exp.route(policy_id)
    challenger_price = challenger_mv.predict([risk_dict])[0]

    # Log the quote (champion prices; challenger is shadow)
    logger.log_quote(
        policy_id=policy_id,
        experiment_name=exp.name,
        arm=arm,
        model_version=champion_mv.version_id,
        quoted_price=champion_price,
        enbp=enbp,
        renewal_flag=renewal_flag,
    )

    return champion_price  # customer always sees champion price

No Radar infrastructure changes required. The library handles the governance layer.

---

ENBP: what the library does and doesn't do

The library records ENBP. It does not calculate it.

ENBP calculation per ICOBS 6B is your pricing team's responsibility. You pass the ENBP value to log_quote(enbp=...). The library records the value, flags whether quoted_price <= enbp, and includes this in the audit report.

If your ENBP calculation is wrong, the log is wrong — but that is upstream of this library's scope. The separation is intentional.

---

Databricks companion notebook

notebooks/insurance_deploy_demo.py demonstrates the full workflow on synthetic data:

• Model registry setup
• Experiment configuration and routing verification
• Quote/bind/claim data generation and logging
• KPI dashboard
• Bootstrap LR test
• ENBP audit report generation

Run as a Databricks Python notebook. Requires pip install insurance-deploy.

---

Scope

This library handles: model version registry, champion/challenger routing, audit logging, KPI computation, statistical tests, ENBP compliance reports.

It does not handle: model training, rate optimisation (see insurance-optimise), model drift monitoring (see insurance-monitor), causal effect estimation (see insurance-causal-policy), or real-time API infrastructure.

---

License

MIT