RiskOn

Rank the functions in a Python codebase by how likely each one is to contain a bug.

What it does

Point RiskOn at a Python project and it produces one ordered list: the functions most likely to contain defects at the top, the least likely at the bottom. It reads your code and your commit history, runs nothing, and changes nothing.

The point is prioritisation. Testing and code-review time is finite, and it usually gets spread evenly across a codebase even though bugs are not spread evenly. RiskOn gives you a defensible order to work through.

Installation

pip install riskon

Requires Python 3.11+. Install git as well — RiskOn works without it, but three of its five signals come from commit history.

Quickstart

riskon doctor          # confirm your environment is set up
riskon predict .       # rank every function in the current project

That's it. The second command prints a ranked table to your terminal.

Usage

riskon predict <path>

Analyses a repository and prints its functions ranked by predicted risk.

Flag	Default	What it does
<path>	required	Project directory to analyse. . is the current one.
--output FILE	nothing written	Also write a JSON report to FILE. Without this, no file is created.
--threshold 0.0-1.0	0.0	Show only functions scoring at or above this — and exit 1 if any function reaches it, so it doubles as a CI gate.
--include-tests	off	Analyse test files and test_-prefixed functions, which are skipped by default.
--config FILE	./.riskon.toml	Read settings from a specific file instead.
--no-color	auto	Force plain output even on an interactive terminal.
--version	—	Print the version and exit.

riskon doctor

Checks your environment and reports problems before you hit them. Read-only:

$ riskon doctor
✓ Python 3.12.3 (>= 3.11 required)
✓ git version 2.43.0 found
✓ Pre-trained model loads correctly
⚠ model was trained with scikit-learn 1.8.0 but 1.9.0 is installed; predictions
  should still be sound, but rerun scripts/train_model.py to remove any doubt
⚠ Current directory has no git history — change_frequency, churn and
  unique_authors will read as 0 if you run 'predict' here

Exits 0 when every check passes or warns, and 1 only on a hard failure — an unsupported Python, or a model that will not load. A scikit-learn version mismatch and a missing git history are warnings, because RiskOn still runs and still produces a useful ranking in both cases.

Configuration file

Drop a .riskon.toml in your project root and stop retyping flags. It is picked up automatically; every key is optional:

# .riskon.toml
threshold = 0.5          # same meaning as --threshold
output = "report.json"   # same meaning as --output
include_tests = false    # same meaning as --include-tests

Precedence, highest first: command-line flag → config file → built-in default.

A malformed file is a hard error — RiskOn will not silently fall back to defaults and hand you a run whose settings nobody chose. An unrecognised key is only a warning: it is named, ignored, and the run continues.

Exit codes

Code	Meaning
0	Ran successfully, and nothing reached the threshold
1	An error occurred, or --threshold was set and something reached it

Both use 1, so a shell cannot tell them apart — but you can, by stream. A tripped gate prints FAIL: N functions at or above ... to stdout; a real error prints Error: ... to stderr. The JSON report is still written when the gate trips, because CI wants the artifact precisely when the build fails.

# GitHub Actions: fail the build if anything crosses 0.7
- run: riskon predict . --threshold 0.7 --output risk.json

As a library

from riskon import RiskOnAnalyzer

result = RiskOnAnalyzer().analyze("./my-project")

for f in result.functions[:10]:          # already sorted, riskiest first
    m = f.metrics
    print(f"{f.predicted_probability:.2f} {f.risk_level.value:<8} "
          f"{m.filepath}:{m.start_line} {m.function_name}")

print(result.summary)
payload = result.to_dict()               # JSON-ready

Worked example

A real run against psf/requests, filtered to 0.40 and above. On a terminal the risk and probability columns are coloured — green for LOW, amber for MEDIUM, red for HIGH and CRITICAL:

$ riskon predict . --threshold 0.4
RiskOn Analysis: /home/you/requests
Showing functions with risk >= 0.40

 File                         Function                   LOC   CC   Risk       Prob
 ──────────────────────────────────────────────────────────────────────────────────
 src/requests/models.py       Response.iter_content       64    7   MEDIUM     0.49
 src/requests/sessions.py     Session.request             97    6   MEDIUM     0.47
 src/requests/sessions.py     SessionRedirectMixin.r…    122   15   MEDIUM     0.45
 src/requests/models.py       PreparedRequest.prepar…     77   19   MEDIUM     0.45
 src/requests/models.py       RequestEncodingMixin._…     69   21   MEDIUM     0.45
 src/requests/utils.py        should_bypass_proxies       61   19   MEDIUM     0.43
 src/requests/__init__.py     check_compatibility         37   10   MEDIUM     0.41
 src/requests/models.py       Response.iter_lines         39    9   MEDIUM     0.41
 src/requests/auth.py         _basic_auth_str             42    5   MEDIUM     0.40

268 functions analyzed, 0 HIGH or CRITICAL

FAIL: 9 functions at or above the 0.40 risk threshold.
$ echo $?
1

Those are, by inspection, some of that library's most tangled and most rewritten functions. Note that nothing reached HIGH: requests is mature and has barely changed in the last year, so the history signals stay small and scores compress downward. On a quiet codebase, read the ordering rather than the bands.

When output is piped or redirected, RiskOn drops to plain text automatically — no colour codes, no box-drawing characters to corrupt a log file.

How it works

For every function RiskOn measures five signals:

• Cyclomatic complexity and lines of code, by parsing the source into an Abstract Syntax Tree. No code is executed.
• Change frequency, churn, and number of distinct authors, from your git history.

The history signals are read per function, not per file, using git log -L <start>,<end>:<file>, which follows a line range backwards through history as edits move it around. That distinction is the point: file-level attribution gives every function in a file the same history, destroying exactly the signal worth having.

Those five numbers — plus three derived from them, to compress heavily skewed distributions — go into a pre-trained Random Forest, which returns a probability between 0 and 1 and a band: LOW below 0.25, MEDIUM below 0.5, HIGH below 0.75, CRITICAL above.

Limitations

The bundled model is trained on synthetic data reflecting relationships reported in the defect-prediction literature, not on labeled real-world defects. No public dataset linking Python functions to the bugs they contained was available, so the training set was generated from a defect-probability function whose coefficients encode those published relationships. The full generating function is written out in scripts/train_model.py and summarised in the model card shipped with the package.

What follows from that:

• The score is a ranking, not a probability. 0.82 does not mean an 82% chance this function is broken. It means the function sits near the top of the risk ordering the model learned. Comparing two scores is meaningful; reading one as a bug rate is not.
• Reported accuracy measures the wrong thing. ROC AUC 0.756 and precision 0.69 at threshold 0.6 describe how well the forest recovered its own generating function on held-out synthetic data. They are not evidence about real defects.
• Mature, low-churn projects compress into the lower bands. Three of the five signals describe recent history; where those are near zero, only complexity and size do any work.
• Renames break history. git log -L does not follow a file across a rename, so a function in a recently renamed file under-reports its history.
• Python only, and the history window is the last 365 days.

Retraining on real labeled data is the intended upgrade path, and needs no code change — the feature order and artifact layout are fixed contracts:

python scripts/train_model.py

Development

git clone https://github.com/aarushi/riskon.git
cd riskon
python -m venv .venv && source .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

pytest                        # 391 tests
pytest --cov=riskon           # coverage report
mypy                          # strict type checking
ruff check src tests scripts  # linting
ruff format src tests scripts # formatting

Tests marked acceptance map one-to-one onto numbered acceptance criteria in docs/PRD.md; tests marked cli drive the installed console script through real subprocesses:

pytest -m acceptance -v
pytest -m "not cli"           # skip the slower subprocess tests

CI runs the full suite on Python 3.11 and 3.12 on Linux, and 3.12 on Windows.

Contributing

Contributions are welcome. Please open an issue before starting substantial work, so the approach can be agreed first.

For pull requests: keep pytest, mypy, and ruff check green, add a test for any behaviour change, and follow the existing conventions — frozen dataclasses for domain models, type hints on every signature, and comments that explain why rather than what. The design decisions behind the current structure, including several approaches that were tried and rejected, are recorded in docs/memory.md; it is worth a skim before proposing anything structural.

License

MIT — see LICENSE.