credonlabs

Model Risk Management SDK for Indian lenders — validation, drift monitoring, explainability, fairness testing and RBI FREE-AI reporting for credit models.

pip install credonlabs

Everything runs locally. The HTTP client is optional and is only needed to push results to the hosted platform.

Handles any model type and any dataset size. Binary, multiclass and regression; scikit-learn, XGBoost, LightGBM, CatBoost, ONNX, PyTorch, or any callable. Datasets stream in constant memory — measured at 11.2 MB peak whether the input is 100 thousand rows or 10 million.

---

Two entry points

	run()	scan()
Data	fits in memory	any size, streamed
Passes	one, in memory	one, batched
Model	required	optional
Stages	all 12	metrics, drift, fairness exact; explainability on a sample

credonlabs.run(model="model.joblib", data="validation.csv", target="default_flag")

credonlabs.scan(data="scores/*.parquet", target="default_flag", scores="score")

scan() with scores= loads no model at all — metrics, drift, calibration and fairness need only what the model produced, not the model itself. That is what makes a 100GB model workable: your scoring layer already wrote the scores, so the SDK reads them.

---

Quick start

import credonlabs

report = credonlabs.validate(
    model=model,                 # any fitted sklearn-compatible estimator
    X=X_val,                     # validation features (DataFrame)
    y=y_val,                     # observed outcomes
    model_name="personal_loan_pd",
    version="1.0.0",
    owner="Risk Team",
    business_purpose="Unsecured personal loan underwriting",
    criticality="tier_1",
)

print(credonlabs.summarise(report))
print(report["verdict"])          # approved | approved_with_conditions | not_approved
print(report["model_card"])       # RBI-inspection-ready Markdown

One call runs every stage. Each is independent: a stage whose inputs are missing reports itself as unavailable rather than aborting the run, so a partial validation still produces usable evidence.

See it work on generated data

credonlabs demo

Generates a labelled dataset with real signal, trains a model, and runs the whole suite. Writes only inside ./credonlabs_demo.

---

What it does

Stage	Requirement	Output
PII boundary	NFR-013	Rejects PAN, Aadhaar, phone, email, IFSC, voter ID
Registry	FR-MRM-001…004	Immutable versioning, criticality tiers, approval workflow
Statistical battery	FR-MRM-010	AUC, KS, Gini, Brier, PSI + bootstrap CIs + calibration
Business-logic tests	FR-MRM-011	Monotonicity, sign and bound constraints
Stability	FR-MRM-012	Perturbation, feature dropout, adversarial noise
Validation report	FR-MRM-013	Signed, inspection-format Markdown
Data drift	FR-MRM-020	Per-feature PSI, alerts at 0.10 / 0.25
Concept drift	FR-MRM-021	AUC/KS decay, 5-point revalidation trigger
Alerting	FR-MRM-022	Per-model thresholds and routing
Explainability	FR-MRM-030/031	TreeSHAP / KernelSHAP, seven feature groups
Counterfactuals	FR-MRM-032	"If X were Y, the score would change by Z"
Bias & fairness	FR-MRM-040/041	Demographic parity, equal opportunity, average odds, incidents
Model card	FR-MRM-050	Purpose, performance, bias, limitations, history
FREE-AI mapping	FR-MRM-051	7 Sutras × 6 Pillars, with honest coverage gaps
Plain-language reasons	FR-MRM-060	Jargon-free borrower explanations

---

Big data and big models

Stream a sharded dataset

report = credonlabs.scan(
    data="s3_export/scores/",      # CSV, Parquet, directory, or glob
    target="default_flag",
    scores="score",                # no model loaded
    baseline="baseline.json",      # enables drift
    protected=["age_band"],
    batch_rows=100_000,
)

Only the columns a stage needs are read. On a wide Parquet table that alone removes most of the bytes on disk.

Reference profiles

Do not re-read the training set every day. Summarise it once:

credonlabs.profile(data="train/", target="default_flag", out="baseline.json")

30,000 rows becomes a 4.8 KB file of bin edges and counts. Daily monitoring compares against that:

credonlabs.compare(data="today/", baseline="baseline.json")

How the streaming metrics stay exact

Every metric decomposes into a single pass:

Metric	Accumulated state	Memory
AUC, KS, Gini	score histogram × {pos, neg}	O(bins)
Brier	running Σ(p−y)²	O(1)
Calibration	per-bin Σp, Σy, n	O(bins)
PSI	bin counts vs stored edges	O(features × bins)
Bias	counters per (group, decision, outcome)	O(groups)
Regression	running sums of error and y²	O(1)

Brier and the regression metrics are exact. AUC from a 10,000-bin histogram matches sklearn.roc_auc_score to six decimal places — verified in tests/test_streaming.py against the same data. The bin count is recorded in every report so a reviewer can see the resolution.

Feeding data in different batch sizes produces bit-identical results, and partial histograms merge(), so parallel readers are safe.

---

Model types

Task	Metrics	Fairness
Binary	AUC, KS, Gini, Brier, calibration	selection-rate parity, equal opportunity, average odds
Multiclass	macro AUC (OvR), accuracy, log loss, confusion	selection-rate parity
Regression	RMSE, MAE, R², MAPE, residuals	prediction parity, group error gaps

The task is detected from the model and the outcome column; the outcome wins, because a classifier scored against a continuous target is a mistake to catch, not a configuration to honour. Override with task="regression".

Fairness changes shape per task deliberately. Selection-rate parity is meaningless for a regression — there the question is whether groups receive systematically different predictions and whether the model is systematically less accurate for one of them.

Model adapters

from credonlabs import SklearnAdapter, CallableAdapter, ScoreColumnAdapter, OnnxAdapter, TorchAdapter

Adapter	For
SklearnAdapter	sklearn, XGBoost, LightGBM, CatBoost wrappers
ScoreColumnAdapter	no model — scores already in the data
CallableAdapter	any function, a queue, a remote endpoint
OnnxAdapter	ONNX Runtime
TorchAdapter	PyTorch, eval mode, batched, device-aware

Chosen automatically; pass one explicitly to override. A score column always wins over a model, because re-running a large model to get numbers you already have is the expensive mistake.

---

The pieces individually

Statistical battery

from credonlabs import statistical

result = statistical.run_battery(
    y_true=y_val,
    y_score=scores,
    baseline_scores=training_scores,   # enables PSI
)

result["auc"]                                  # 0.9810
result["confidence_intervals"]["auc"]          # {'lower': 0.97, 'upper': 0.99, ...}
result["calibration"]["mean_absolute_calibration_error"]

Business-logic constraints

Catches the defect a purely statistical battery cannot see: a model can hold an AUC of 0.85 while still lowering creditworthiness as income rises.

from credonlabs import MonotonicConstraint, SignConstraint, BoundConstraint

constraints = [
    MonotonicConstraint(feature="verified_monthly_income", direction="decreasing"),
    SignConstraint(feature="emi_bounces_12m", expected_sign="positive"),
    BoundConstraint(feature="credit_score", minimum=300, maximum=900),
]

report = credonlabs.validate(..., constraints=constraints)

direction="decreasing" means higher income must not increase the probability of default. Violations come back with severity and the exact inputs that triggered them.

Bias audit

report = credonlabs.validate(
    ...,
    protected_attributes={"age_band": validation["age_band"]},
)

report["fairness"]["attributes"]["age_band"]["disparate_impact_ratio"]   # 0.254
report["fairness"]["incidents"]                                          # incident records

Ratios outside 0.80–1.25 (the four-fifths rule) raise an incident with affected groups, sample sizes, and the FR-MRM-041 SLA.

Protected attributes must arrive already banded — age_band, not a date of birth.

Explainability and borrower reasons

local = credonlabs.explain_local(model, X_val, row=0)

reason = credonlabs.generate_reason(
    group_attribution=local["group_attribution"],
    decision="declined",
)

print(reason["text"])
print(reason["jargon_check"])    # {'clean': True, 'banned_terms_found': []}

We were not able to approve this application. The main reasons were that there were missed or late repayments in your recent credit history and you are already repaying a large amount each month relative to what you earn. In your favour, your income was steady and at a comfortable level. You can ask us to look at this decision again if you think something has been recorded incorrectly.

Every generated sentence is checked against a banned-terms list — no "SHAP", "percentile", "model", "probability". Templates are data, not code: register a translated set with reasons.register_locale() so the lender's legal team owns the wording.

Counterfactuals

result = credonlabs.generate_counterfactuals(model, row=0, X=X_val, top_n=3)

for entry in result["counterfactuals"]:
    print(entry["statement"])

Searches single-feature, policy-compliant changes only. Age band, pincode tier, gender and employment category are never suggested, whatever the caller passes.

Monitoring

report = credonlabs.run_monitoring(
    baseline_features=X_train,
    current_features=X_production,
    baseline_scores=scores_train,
    current_scores=scores_production,
    current_outcomes=y_production,        # once outcomes arrive
    baseline_metrics={"auc": 0.98, "ks": 0.89},
)

report["alert_level"]                      # ok | warning | critical
report["data_drift"]["drifted_features"]
report["concept_drift"]["revalidation_required"]

The three signals become available at different times — features first, scores next, outcomes last — so each section runs only when its inputs are present.

Registry

store = credonlabs.ModelRegistry("registry.json")

store.register(
    model_name="personal_loan_pd",
    version="1.0.0",
    owner="Risk Team",
    criticality="tier_1",
    business_purpose="Underwriting",
    artifact_path="model.joblib",     # records SHA-256 + size
)

store.approve("personal_loan_pd", "1.0.0", "priya")
store.approve("personal_loan_pd", "1.0.0", "ravi")   # tier_1 needs two distinct approvers

store.diff("personal_loan_pd", "1.0.0", "1.1.0")     # what changed between versions

Versions are immutable — re-registering the same version is refused. Tiers can be escalated but never silently downgraded. Deprecation never deletes.

---

Command line

credonlabs validate --model model.joblib --data validation.csv \
    --target default_flag --model-name personal_loan_pd --version 1.0.0 \
    --owner "Risk Team" --purpose "Underwriting" \
    --protected age_band --card model_card.md --out report.json

Exits non-zero when the verdict is not_approved, so it gates a CI pipeline directly.

credonlabs monitor --model model.joblib --baseline train.csv \
    --current production.csv --target default_flag

credonlabs registry --registry registry.json --csv inventory.csv

credonlabs demo

---

Platform client (optional)

pip install credonlabs[http]

from credonlabs import CredonClient, ClientConfig

client = CredonClient(ClientConfig(
    api_key="...",
    lender_tenant_id="nbfc_alpha",
))

client.register_model(record.to_dict())
client.submit_validation(model_id, report)
client.score(features={...}, product_code="personal_loan_unsecured")

client.score() runs the PII scrubber before the payload leaves the process. Platform errors come back as the same exception classes the SDK raises locally, so you handle one taxonomy:

Exception	Code	HTTP
SchemaValidationError	SCHEMA_VALIDATION_ERROR	400
PIIDetectedError	PII_DETECTED	400
UnauthorizedError	UNAUTHORIZED	401
TenantAccessDeniedError	TENANT_ACCESS_DENIED	403
IdempotencyConflictError	IDEMPOTENCY_CONFLICT	409
CoverageTooLowError	COVERAGE_TOO_LOW	422
RateLimitedError	RATE_LIMITED	429
ModelUnavailableError	MODEL_UNAVAILABLE	503

---

Design notes

No PII, ever. pii.assert_clean() rejects a feature vector before it leaves your process. Errors name the field, never the value — an error payload can't itself become a leak. Measured floats are exempt from the numeric patterns: the digits of -0.1321048632913019 contain a ten-digit run that looks like a mobile number, and flagging it would make the scanner useless.

Explainability-first. The SDK never returns a score with no attribution. Without shap it falls back to native importances or standardised coefficients and marks the result degraded with the reason — it doesn't silently return nothing.

Honest reporting. freeai.coverage_for_report() marks an obligation as evidenced only if the stage that evidences it actually ran. A validation that skipped the bias audit reports S4 as not evidenced, so a lender never claims coverage it hasn't demonstrated. The model card's Limitations section is populated from real findings, not boilerplate.

Verdicts are blocking. PII in the data, a high-severity business-logic violation, a high-severity bias incident, or no attribution at all all produce not_approved for a tier-1 model — not a warning.

---

Publishing reports to Credon

credonlabs.run(..., api_key="mrm_...")     # or set CREDONLABS_API_KEY

Reports post to https://api.credonlabs.com — the endpoint ships with the package, so no integrator's code names a URL. Override with base_url= or CREDONLABS_BASE_URL for staging or a self-hosted engine.

The key is an MRM key, created from the dashboard's API keys page or POST /mrm/keys. It is separate from the credit-model scoring key: leaking the scoring key must not expose validation reports.

Nothing is sent unless a key is supplied.

Install options

pip install credonlabs            # core: numpy, pandas, scikit-learn, scipy
pip install credonlabs[shap]      # + SHAP attributions
pip install credonlabs[parquet]   # + Parquet streaming (pyarrow)
pip install credonlabs[onnx]      # + ONNX Runtime scoring
pip install credonlabs[http]      # + platform client
pip install credonlabs[all]
pip install credonlabs[dev]       # + pytest, build

Requires Python 3.10+.

Tests

pytest tests/ -q