graphnetz 0.2.0

Statistically rigorous benchmarking for graph neural networks.

graphnetz trains every (task, model, seed) triple through one pipeline and returns a statistical report instead of an accuracy table:

• a Student's t confidence interval for every cell,
• paired t-tests or Wilcoxon signed-rank tests within each task, Holm-corrected,
• Friedman ranks and a Nemenyi critical difference across tasks,
• power, minimum detectable effect and equivalence tests, so a non-significant result can be read as "tied" or "underpowered".

The catalogue holds 62 dataset loaders across 10 domains and 4 task types, with 5 architectures and a DGI pre-training utility.

Documentation: kleyt0n.github.io/graphnetz

Install

uv add graphnetz                # core
uv add "graphnetz[ogb]"         # plus the OGB loaders
uv add "graphnetz[chem]"        # plus RDKit, for the molecular loaders

Requires Python 3.10+, torch 2.6+ and torch-geometric 2.6+.

For development:

git clone https://github.com/kleyt0n/graphnetz
cd graphnetz
uv sync --group dev

Quick start

from graphnetz import GAT, GCN, GraphSAGE, run_benchmark

report = run_benchmark(
    "social",
    {"GCN": GCN, "GAT": GAT, "GraphSAGE": GraphSAGE},
    seeds=range(10),
    task_type="node_cls",
)

report.summary()                        # mean and t-CI per (task, model)
report.pairwise()                       # Holm-corrected paired tests
report.friedman()                       # omnibus test on ranks across tasks
report.power()                          # what this design could detect
report.plot_critical_difference()       # Demšar diagram
report.to_latex("results.tex")          # booktabs table, row-best in bold

The report

run_benchmark returns a BenchmarkReport. Every method reads the same X[task, model, seed] tensor, so nothing downstream retrains.

See Reading the report.

Tasks

Unlabelled graphs (Netzschleuder, synthetic combinatorial, Ising lattice) enter through link prediction on a held-out edge split, so every cell carries a held-out metric.

Datasets

† Requires the ogb extra. OGB loaders live in their domain category.

from graphnetz import Netz
from graphnetz.datasets.social import cora

ds = cora("data/cora")
streets = Netz(root="data", dataset_name="urban_streets", network_name="brasilia")

Netz loads any network from the Netzschleuder archive. See Datasets.

Models

Node-level encoders reach graph-level and link tasks through adapters the runner attaches automatically.

Custom models

A model needs __init__(in_channels, hidden_channels, out_channels), forward(data) and a declared task type. It then runs with the same seeds, splits and corrections as the built-ins.

import torch
from graphnetz import GCN, register_model, run_benchmark


@register_model(task_type="node_cls")
class MyGNN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels): ...
    def forward(self, data): ...


report = run_benchmark("social", {"GCN": GCN, "MyGNN": MyGNN}, seeds=range(10))

Task types can also be declared with a task_types class attribute or an inline (cls, task, factory) tuple. Custom datasets go in through run_benchmark(tasks=[Task(...)]). See Custom models and Custom datasets.

Examples

Citation

@misc{dacosta2026graphnetz,
  title         = {{GraphNetz}: Statistical Benchmarking of Graph Neural Networks with Paired Tests and Rank Aggregation},
  author        = {Kleyton da Costa and Bernardo Modenesi},
  year          = {2026},
  eprint        = {2605.09099},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CE},
  url           = {https://arxiv.org/abs/2605.09099}
}

License

MIT. See LICENCE.txt.