cve-risk-engine

Calibrated random forest scoring of CVE exploitability, packaged with the honest limits of the study it comes from.

This is the reference implementation of the scoring layer of

Exploitability-Driven CVE Prioritization for Security Operations: A Recursive Feature Elimination Random Forest Approach — Reza Yuzron Wardana, Hari Purnama. Manuscript in preparation. Code, data and evaluation artifacts: CVE-RiskEngine-RF

What this package is not. It is not a claim that machine learning beats EPSS. On the study's held-out temporal test partition the forest ranks better than disclosure-time EPSS, but the difference in average precision is not statistically established, the raw scores are not probabilities, and a CVSS-only score stays close behind. The package ships those facts as metadata and refuses to return an uncalibrated number as a probability.

Install

pip install CVE-RiskEngine-RF
# optional HTTP service
pip install "CVE-RiskEngine-RF[api]"

Use

from cve_risk_engine import CveRiskEngine

engine = CveRiskEngine.load()
result = engine.score(
    cvss_vector="CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
    cvss_score=9.8,
    epss_score=0.00061,   # EPSS at or before the disclosure date, not today's value
    cve_id="CVE-2025-14611",
    cwe="CWE-798",
)

{
  "cve_id": "CVE-2025-14611",
  "raw_model_score": 0.897937,
  "calibrated_probability": 0.012108,
  "cvss_score": 9.8,
  "epss_score": 0.00061,
  "final_risk_score": 0.4961,
  "risk_level": "medium",
  "recommendation": "Patch within 30 days",
  "features": {...}
}

Command line:

cve-risk-engine info
cve-risk-engine score --vector "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" \
                      --cvss 9.8 --epss 0.00061 --cwe CWE-798 --id CVE-2025-14611
cve-risk-engine batch records.csv --out scored.csv

HTTP:

uvicorn cve_risk_engine.api:app --port 8000
# GET /health  GET /model  POST /score  POST /score/batch

Read the two numbers differently

What the model actually does

Corpus: 44,474 CVEs published in 2025 with a CVSS v3.1 vector. Target: CISA KEV entry within 90 days of disclosure, 109 positives (0.245%); CVEs already KEV-listed at disclosure are excluded, so no record is right-censored. Temporal split: train 25,951 (70 positives) to 2025-07-31, validation 7,363 (18) Aug–Sep, test 11,160 (21) Oct–Dec. Features: 7 of 13 candidates retained by RFECV on the training partition. Imbalance: SMOTE 1:1 chosen on validation, applied inside the training folds only.

Ranking differences on average precision and MCC are inside their bootstrap intervals. The recall gain is real but comes at roughly twice the queue length.

KEV membership is not a risk band, and high/critical are unreachable

A natural question is whether a CVE that is listed in CISA KEV comes back as high or critical. It does not, and the engine never accepts KEV as an input: KEV is the evaluation label of the study, never a feature, so a served decision is reproducible from the model alone. On the test partition the 21 KEV-listed CVEs come back as 15 medium and 6 low, none above medium, with final_risk_score between 0.3058 and 0.5061. A KEV CVE at CVSS 7.5 lands in low; so does one at CVSS 6.1.

The reason is the one in the second limitation below: at a 0.245% base rate, calibrated_probability is at most a few percent, so FinalRisk is dominated by the CVSS term and saturates around 0.53. Read the band as a severity-flavoured label, not as an exploitation verdict, and never as a KEV oracle. Rank by final_risk_score and cut by queue capacity; that is the operating point the study evaluates.

Two limitations you must design around

1. Raw scores are not probabilities. They are ranked correctly but are badly calibrated (Brier 0.0629). Always read calibrated_probability, never raw_model_score, when a number is shown to a human.

2. The high and critical bands are unreachable. The bounds come from the CVSS v3.1 severity scale on the unit interval, but once the score is calibrated on a 0.245% base rate, final_risk_score cannot exceed about 0.53 even at CVSS 10.0. On the test partition the high and critical bands hold zero records; everything lands in low (8,421 records, 6 exploited) or medium (2,739 records, 15 exploited). Precision still rises monotonically between them, but a banding scheme usable under calibrated probabilities has to be defined on validation quantiles of the score, not on the CVSS scale. For that reason the metadata ships capacity_cutoffs_validation (the score above which the top 0.5%, 1%, 2%, 5% and 10% of the validation backlog fall) and the package README recommends cutting by capacity. Treating risk_level as an absolute severity rating is wrong.

Blended-score sensitivity on the calibrated probability, for reference: test average precision rises with the model weight (0.0295 at 0.5/0.5, 0.0331 at 0.7/0.3, 0.0347 at 0.8/0.2) against 0.0159 for the model alone and 0.0102 for CVSS alone, while the operational cost of a 500-record queue is identical (1,590, with 10 of the 21 exploited CVEs caught) for every blend from 0.5 upward. Equal weights are used because they sit in the middle of that flat region.

Bundled artifact

cve_risk_engine/artifacts/ holds the classifier, the Platt map, the CWE frequency map fitted on the training partition, and a metadata file recording the hyperparameters, the target definition, the split windows, the test metrics, the paired intervals, the input SHA-256 digests and the limitation above. The artifact reproduces the manuscript's test numbers exactly (average precision 0.015903, ROC-AUC 0.880724), and tests/reference_scores.csv freezes 821 test records including all 21 positives so anyone can re-verify that claim without the study repo:

python -m pytest tests -q

Rebuilding the artifact

The artifact is exported from the study pipeline, never hand-edited:

python scripts/export_artifact.py \
    --study-repo /path/to/CVE-RiskEngine-RF \
    --out src/cve_risk_engine/artifacts
python scripts/build_reference_scores.py \
    --study-repo /path/to/CVE-RiskEngine-RF \
    --out tests/reference_scores.csv

The first script asserts that the exported model's test average precision and ROC-AUC are bit-identical to revision/outputs/results.json; if they drift, the export fails rather than shipping a lookalike.

Licence

MIT. The bundled artifact is trained on derived metadata from NVD, CISA and FIRST; the upstream terms of those sources apply to the data.