SigReview

Deterministic code review for data science, ML and GenAI projects. One install, one command, one report.

pip install sigreview
sigreview scan -f ./my_project                      # a local folder
sigreview scan -r https://github.com/org/repo       # a Git repo link (GitHub, GitLab, Azure DevOps, Bitbucket)

SigReview reads every .py, .ipynb and Databricks notebook in the project, runs its own 90 data science rules plus six proven tools, and writes four files into a scan_result folder:

• sigreview_report.html: the report for people
• sigreview_findings.json: the same findings for machines (CI, the review agent)
• sigreview_findings.sarif: for inline findings on GitHub and Azure DevOps pull requests
• sigreview_summary.md: a short summary to post as a PR comment

Nothing in your project is executed. SigReview only reads code.

Test files (test_*.py, *_test.py, conftest.py, test/ and tests/ folders) are not checked by the data science rules or the type checker: tests break rules on purpose.

Commands

Command	What it does
sigreview scan -f "folder"	Scan a local folder. Reports go to folder/scan_result/
sigreview scan -r "repo link"	Clone a repo, scan it, delete the copy. Reports go to ./scan_result/
sigreview baseline -f "folder"	Record today's findings as known, so later scans show only new ones
sigreview doctor	Check that every wrapped tool and Git are installed, with versions
sigreview rules [ID]	List every rule, or show one in full: sigreview rules DS001
sigreview --version	Show the version

Options for scan:

Option	What it does
-b, --branch NAME	With -r: the branch to scan. Default: the repo's default branch
-d, --diff BRANCH	Only show findings in files changed since that branch, e.g. --diff main
--fail-on LEVEL	high (default), medium, low or none: the lowest severity that fails the scan
--baseline FILE	Baseline file to use. Default: .sigreview-baseline.json in the project, if present
-o, --output FOLDER	Write the reports somewhere other than scan_result/

scan exits with 1 when a finding reaches the --fail-on level, so it can gate CI, and 2 on usage errors.

Severity	Meaning
High	Proven problem. Fix before merging.
Medium	Likely bug or risky pattern. Someone should look.
Low	Cleanup and readability.

Scanning a repo link

-r works with GitHub, Azure DevOps, Bitbucket and GitLab (and any other Git host), with https://... or git@... links:

sigreview scan -r https://github.com/org/churn-model
sigreview scan -r git@github.com:org/churn-model.git --branch feature/new-features
sigreview scan -r https://dev.azure.com/org/project/_git/routing
sigreview scan -r https://bitbucket.org/team/forecast

Login. Public repos need nothing. For private repos:

1. Your own Git login is used by default: whatever already works for git clone on your machine (saved HTTPS login, Git Credential Manager, or your SSH key for git@ links).
2. A token, for CI or a machine without a saved login: set SIGREVIEW_GIT_TOKEN. It is used instead of your own login when set.
3. SIGREVIEW_GIT_USER only if your host needs a specific username with the token. The defaults work for GitHub, Azure DevOps, Bitbucket and GitLab tokens.

Host	Token to create	Access it needs
GitHub	Fine-grained personal access token	Contents: Read-only, on the repos to scan
Azure DevOps	Personal access token (PAT)	Code: Read
Bitbucket	Repository access token, or API token	Repositories: Read
GitLab	Personal or project access token	read_repository

export SIGREVIEW_GIT_TOKEN=...          # Windows PowerShell: $env:SIGREVIEW_GIT_TOKEN="..."
sigreview scan -r https://dev.azure.com/org/project/_git/routing

Inside a CI pipeline the platform's own token is picked up automatically: GITHUB_TOKEN, SYSTEM_ACCESSTOKEN (Azure Pipelines), BITBUCKET_TOKEN or CI_JOB_TOKEN (GitLab).

The token is never accepted on the command line (it would end up in your shell history), is passed to Git without being written to disk, and is masked in every message.

Reviewing pull requests

sigreview scan -f . --diff main                     # only files changed since main
sigreview scan -f . --diff origin/main --fail-on medium

Old projects. Run sigreview baseline -f . once and commit .sigreview-baseline.json. Every later scan hides those known findings and shows only new ones. Run it again whenever you want to accept the current state.

What it checks

Data science rules (SigReview's own, run sigreview rules for the full list)

Area	Rules
Machine learning	DS001-DS015: leakage before the split (sklearn and pandas), fitting on test data, target left in features, target-based features, resampling before the split, early stopping on the test set, the same entity in train and test, separate encoding of train and test, unscaled features for KNN/SVM, missing seeds, PyTorch zero_grad / eval() / unsafe torch.load
Evaluation	EV001-EV004: probability metrics on hard labels, swapped metric arguments, model chosen on the test set, scoring on training data only
Time series	TS001-TS007: shuffled splits, windows that include the current target, centred windows, backward fill, future values as features, no gap for multi-step forecasts, scoring on the tail of the fitted series
Deep learning	DL001-DL007: softmax before a logits loss, loss tensors accumulated, wrong step order, softmax without dim, unshuffled training loader, model.forward(), fine-tuning evaluated on its training data
DataFrames	DF001-DF014: chained assignment, results thrown away, iterrows() writes, pandas 2.0 removals, merge without validate=, concat in a loop, inplace=True assigned, index misalignment, .loc[i] after filtering, pandas 3 string and datetime changes, ignored conversion errors, ambiguous dates, NaN turned into "nan"
Spark	SP001-SP008: collect() / toPandas() without limit(), withColumn in a loop, == None, UDFs without a return type, static partition overwrite, actions in a loop, monotonically_increasing_id as row number, single-partition writes
Notebooks	NB001-NB008: names used before the cell that defines them, out-of-order runs, !pip install, saved errors, large outputs, data tables and secrets saved in outputs, %run
GenAI and LLM apps	LLM001-LLM019: model output run as code, SQL or HTML, agents with shell or code tools, remote code from model hubs, unsafe pickles, user text in the system prompt, whole DataFrames in prompts, unguarded JSON parsing, no output limit, no timeout or unbounded retries, one call per row, unpinned model versions, mixed embedding models, unbounded chat history, removed SDK APIs, prompts in logs, right padding for generation, chunks longer than the embedding model
Project	PR001-PR008: no dependency file, unpinned versions, no tests, large data files or secret files committed, hardcoded absolute paths, training window tied to today's date, models saved without library versions

Wrapped tools (installed with SigReview)

Tool	Checks
Ruff	Likely bugs, risky security patterns, unused imports and variables, functions with a complexity above 30
Pyrefly	Type errors: wrong arguments, missing attributes
Vulture	Functions and classes nothing calls, unreachable code
deptry	Imports missing from the dependency file, unused dependencies
pip-audit	Known vulnerabilities in pinned packages (needs internet)
detect-secrets	API keys, tokens and passwords in files

Style nits do not get one finding per line. Each kind is reported once for the whole project, with a count and the files most affected: formatting, docstrings and legacy-random.

Databricks. Notebooks exported to Git as .py files (they start with # Databricks notebook source) are read as notebooks: the notebook rules run, and every finding shows its cell number.

Silencing a finding

On the line itself:

df = pd.read_csv("/Users/anna/data.csv")  # sigreview: ignore[PR006]

For the whole project, in pyproject.toml:

[tool.sigreview]
ignore = ["PR003", "docstrings"]    # rule IDs to switch off
exclude = ["legacy/", "*_old.py"]   # paths to skip

Use in CI

Ready-made setups are in the ci/ folder: copy the one for your platform.

GitLab (ci/gitlab-ci.yml): scans the files changed in each merge request and keeps the reports as job artifacts.

Azure Pipelines (ci/azure-pipelines.yml), GitHub Actions (ci/github-actions.yml) and Bitbucket (ci/bitbucket-pipelines.yml) work the same way.

Any CI, in two lines:

pip install sigreview
sigreview scan -f . --diff origin/main --fail-on high

pre-commit, with sigreview installed in your environment:

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: sigreview
        name: sigreview
        entry: sigreview scan -f . --diff main
        language: system
        pass_filenames: false
        always_run: true
        stages: [pre-push]

Add scan_result/ to .gitignore in projects you scan.

Colour and symbols switch off automatically in CI logs, pipes and when NO_COLOR is set.

Development

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pytest
sigreview scan -f tests/fixtures/sample_project

Try it on the sample project, then open tests/fixtures/sample_project/scan_result/sigreview_report.html in a browser. The sample has one or more examples of most rules.

Colours live in src/sigreview/theme.py: SIGMOID_RED for the terminal, REPORT_COLOURS for the report. The report embeds the Red Hat Text and Red Hat Mono fonts (SIL Open Font License, see src/sigreview/report/templates/fonts/OFL.txt), so it looks the same offline and on every machine.

Adding a rule: write a function with the @rule(...) decorator in the right file under src/sigreview/rules/, then add tests/fixtures/rules/<ID>_bad.py and <ID>_good.py. The tests check that every rule fires on its bad file and stays silent on its good one.

Author: Subhabrat Biswal