IDS²Eval

hashtagRR.github.io/IDS2Eval

A config-driven toolkit that audits network intrusion detection (NIDS) datasets before you trust a result on them. It runs 28 checks for duplicated rows, label conflicts, one-column shortcuts and split problems, tests whether a suspected problem actually changes the accuracy, and records every verdict in a citable scorecard.

IDS² reads both ways: Intrusion Detection Systems and Intrusion Data Sets.

Quickstart

python3 -m venv venv && venv/bin/pip install "ids2eval[plots]"
venv/bin/ids2eval-dashboard                      # web UI with a pre-filled config editor
venv/bin/ids2eval run --config my_config.yaml    # or the CLI, with a config like the one below

# my_config.yaml: any NIDS dataset in CSV or Parquet form
dataset:
  name: my-dataset
  raw_files: ["data.csv"]
schema:
  label_column: Label

Windows, installing from the source code, and troubleshooting: INSTALL.md.

Documentation

Versions

Every release and its changes are in the changelog; each one is archived on Zenodo under the concept DOI above.

• v0.2.1 (10.5281/zenodo.23301210): a check with no input to test reports not_applicable instead of ok, an audit in which no check produced a result is not_evaluated instead of passed, scorecards list the checks a config left disabled (schema 1.4), and known-issue warnings name the release they document; no statistic or threshold changed. Also adds the validation scripts and results of the paper's revision (host-disjoint splits, workflow ablation, interval coverage, class-adjusted reuse, a comparison with Flood et al.'s released heuristics).
• v0.2.0 (10.5281/zenodo.23217301): the code, configurations and results behind the accompanying paper on auditing NIDS benchmarks (in preparation).

IDS²Eval grew out of the dataset-auditing code of research on machine-learning-based network intrusion detection, generalized into a dataset-independent toolkit.

Citing

Cite the version you used. The concept DOI 10.5281/zenodo.23217300 always resolves to the latest version.

Relapanawa, R. (2026). IDS2Eval 0.2.1 (Version v0.2.1) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.23301210

The results, configurations and analysis scripts behind the accompanying paper are archived separately:

Relapanawa, R. (2026). IDS2Eval paper: research artifacts (results, configurations and analysis scripts) [Dataset]. Zenodo. https://doi.org/10.5281/zenodo.23301258

License

AGPL-3.0. See LICENSE.