Mule-Hunt

Mule-Hunt is a graph-neural-network (GNN) pipeline for detecting coordinated fraud in UPI-style payment networks.

The project represents payment activity as a graph:

• an account is a node;
• a transfer is a directed edge; and
• a planted fraud ring is a group of accounts that move money in a cycle.

The model predicts which accounts belong to a fraud ring. This lets the pipeline evaluate network-level signals—such as unusual connectivity and coordinated cycles—that are invisible when each transaction is considered in isolation.

Research/demo project: all experiments use synthetic data. The scores in this repository are not production fraud-detection claims and must not be used to make decisions about real accounts.

What is included

• Synthetic UPI-style graph generation using SantanderAI/gen-fraud-graph.
• A small built-in generator for tests and offline smoke runs.
• CSV-to-PyTorch Geometric graph loading.
• Ring-aware train/validation/test splits that hold out complete fraud rings.
• Three GNNs: GCN, GraphSAGE, and GATv2 — with configurable depth (message-passing layers) and Jumping-Knowledge aggregation instead of a hard-coded 2-hop receptive field.
• A transaction-level head on the same backbone: per-transfer risk scores from node embeddings + amount/time features, trained jointly with the node loss.
• Non-graph baselines: Random Forest, HistGradientBoosting, and optional XGBoost.
• Leak-aware node feature construction and class-imbalance handling.
• Isotonic probability calibration with validation-brier reporting.
• A cold-start fallback (gradient-boosted tabular model) for low-activity accounts, whose neighborhoods are too small for a GNN to score.
• Population-stability-index (PSI) drift monitoring between train and test score distributions.
• AUC, average precision, and whole-ring recovery metrics.
• A FastAPI risk-scoring service with a lightweight dashboard.
• Optional plain-language account explanations, with a deterministic local fallback when no API key is configured.

Dashboard

upifraud serve runs the FastAPI backend plus a browser dashboard: summary metrics, the top-50 highest-risk accounts, a risk-score histogram, ring exploration on a live graph, per-account neighborhoods, and plain-English risk explanations (optionally generated by an LLM via OPENAI_API_KEY, with a deterministic local fallback).

Explanations are model-grounded: GNNExplainer runs on the account's 2-hop neighborhood and reports which features and which neighbors drive the risk score:

How the pipeline works

synthetic CSV graph
        │
        ▼
account and transaction tables
        │
        ▼
PyG Data object
  ├── node features: account + connectivity
  ├── directed transaction edges
  ├── fraud-ring labels
  └── ring-aware train/validation/test masks
        │
        ├── GraphSAGE / GCN
        └── tabular baselines
                │
                ▼
       held-out evaluation + risk scores
                │
                ▼
        FastAPI service + dashboard

Requirements

• Python 3.11 or newer
• Git, because the synthetic graph generator is installed from GitHub
• uv is recommended for environment and dependency management
• A CPU is sufficient for the included demo and tests

Installation

Using uv:

git clone https://github.com/sohamvjadhav/Mule-Hunt.git
cd Mule-Hunt

uv venv --python 3.12
source .venv/bin/activate       # Windows PowerShell: .venv\Scripts\Activate.ps1
uv pip install -e ".[dev]"

Without uv:

python3 -m venv .venv
source .venv/bin/activate       # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

Verify the installation:

upifraud --version
pytest

Quick start

Run the complete pipeline on a small synthetic graph:

upifraud demo --toy --rings 5

This command generates a toy graph, trains GraphSAGE, trains Random Forest and HistGradientBoosting baselines, evaluates all saved models, and prints the highest-scoring accounts. It writes generated data to data/raw and model artifacts to models.

For a small graph produced by the external generator instead of the built-in toy generator:

upifraud demo --data data/external --out-dir models-external \
  --scale 0.0001 --rings 10

After training, start the API and dashboard:

upifraud serve --out-dir models

Open http://127.0.0.1:8000 in a browser. The API is also available at:

curl http://127.0.0.1:8000/healthz
curl http://127.0.0.1:8000/risk/account/acc_42

curl -X POST http://127.0.0.1:8000/risk/batch \
  -H 'Content-Type: application/json' \
  -d '{"account_ids": ["acc_42", "acc_99"]}'

The account IDs depend on the generated dataset, so replace acc_42 and acc_99 with IDs that exist in your graph.

Command-line reference

The package exposes one command, upifraud, with the following subcommands.

Generate data

upifraud generate --output data/raw --scale 0.001 --rings 50

Use the built-in generator for a fast, deterministic smoke run:

upifraud generate --toy --output data/toy --toy-accounts 300 \
  --toy-tx 2500 --rings 5 --seed 42

Important generation options:

Option	Default	Description
--output	data/raw	Directory for graph CSVs
--scale	0.001	Scale passed to gen-fraud-graph
--rings	generator default	Number of planted fraud rings
--hardness	low	Synthetic difficulty: low, medium, or high
--workers	1	Generator worker count
--toy	off	Use the built-in generator
--seed	42	Seed for the toy generator

The generated directory contains accounts/, transactions/, and fraud/.

Train a GNN

upifraud train-gnn \
  --data data/raw \
  --out-dir models \
  --model sage \
  --epochs 200 \
  --test-rings 3

Available GNNs are sage, gcn, and gat. The default rings split holds out whole rings for testing. Use --split random only when you specifically want a random node split for comparison; it is less representative of discovering a previously unseen ring.

All three architectures are configurable in depth: --num-layers (default 2) stacks message-passing layers and Jumping Knowledge (--jk cat, the default, or max) concatenates every layer's embeddings before the classifier, so the receptive field is 1..N hops and deeper stacks avoid over-smoothing. Use --edge-loss-weight 0 to drop the transaction-level head and train node-only (the edge head is trained jointly by default with weight 0.5).

The command saves:

• <model>.pt: model weights;
• <model>_args.json: model dimensions, feature standardization values, validation brier before/after calibration, and cold-start metadata;
• <model>_calib.pkl: isotonic calibrator fitted on validation scores;
• <model>_coldstart.joblib: cold-start tabular model and metadata; and
• graph.pt: the processed graph and its split masks.

Calibration and cold-start are on by default and can be adjusted:

Option	Default	Description
--calibrate / --no-calibrate	on	Fit an isotonic calibrator on validation scores
--cold-start-threshold	10	Combined degree below which a tabular cold-start score is used instead of the GNN
--num-layers	2	Message-passing depth; Jumping Knowledge aggregates all layers
--jk	cat	JK aggregation: cat or max
--edge-loss-weight	0.5	Weight of the transaction-level loss (set to 0 to disable the edge head)

Train a baseline

upifraud train-baseline \
  --data data/raw \
  --out-dir models \
  --model rf

Available baselines are rf, hgb, and xgb. Install the optional XGBoost dependency before using xgb:

python -m pip install -e ".[dev,xgb]"

Compare saved models

upifraud evaluate --out-dir models

This reads the saved graph and model artifacts and prints AUC, average precision, mean ring recall, and fraud hit rate at the evaluation cutoff.

Run the benchmark matrix

upifraud benchmark \
  --root bench \
  --scale 0.001 \
  --rings 50 \
  --test-rings 10

The benchmark trains the selected GNN and baselines at each requested hardness level and writes bench/results/benchmark.json. Add --regenerate to replace already-generated benchmark data.

Data and features

The external generator is the open-source synthetic gen-fraud-graph project, used under Apache-2.0. No real financial data is included or required.

Mule-Hunt loads account CSVs, transaction CSVs, fraud transaction labels, and fraud-case metadata into one PyG graph. Each account receives a node label of 1 when it belongs to a planted ring and 0 otherwise.

By default, the node representation uses account and structural features:

• log balance;
• account risk score;
• account age;
• in-degree and out-degree; and
• unique inbound and outbound counterparties.

Constant columns are removed before training. --amount-stats adds inbound and outbound amount aggregates, but those features are intentionally excluded by default: the synthetic generator uses a distinctive amount for planted ring transactions, so amount aggregates can reveal the label through a benchmark artifact rather than through network structure.

--cycle-counts adds per-node triangle counts and the local clustering coefficient (undirected 3-cycle structure, computed in milliseconds on the 10k-node graph via sparse adjacency multiplication). The experiment is documented below: on gen-fraud-graph data, 32% of ring nodes sit in at least one triangle (vs 24% of normal nodes), but the overlap is large and the features did not help the GNN.

Evaluation protocol

The default evaluation is designed to test whether a model can find new rings:

1. Fraud rings are split as groups, not as individual accounts.
2. All members of a held-out ring stay in the same test split.
3. Normal accounts are sampled into train, validation, and test splits.
4. Training uses class-weighted binary cross-entropy and early stopping on validation average precision.
5. Results are reported on the held-out test accounts.

The main metrics are:

• AUC: ranking quality across positive and negative accounts;
• average precision (AP): more informative than accuracy for the imbalanced fraud labels;
• mean ring recall: the average fraction of each held-out ring recovered in the top-k ranked accounts, where k is the test-set size; and
• operating point: precision/recall at the F1-maximizing score threshold on held-out rings — the number an investigator would actually deploy with.

Current benchmark results

The committed benchmark uses approximately 10,000 accounts, 90,000 transfers, 50 rings, and 10 held-out rings. Runs include the transaction-level head (edge AUC is reported on held-out transactions). The generator has its own randomness, so numbers may vary slightly between runs.

Hardness	Model	Layers	AUC	AP	Mean ring recall	Edge AUC
low	GraphSAGE	2	0.620	0.056	0.238	1.000
low	GraphSAGE	3	0.630	0.058	0.273	1.000
low	Random Forest	–	0.507	0.036	0.142	–
medium	GraphSAGE	2	0.671	0.079	0.359	0.999
medium	GraphSAGE	3	0.610	0.051	0.324	0.999
medium	Random Forest	–	0.603	0.065	0.269	–
high	GraphSAGE	2	0.665	0.069	0.351	0.997
high	GraphSAGE	3	0.629	0.044	0.262	0.997
high	Random Forest	–	0.622	0.057	0.185	–

Full results are in results/benchmark.json. The important comparisons: Random Forest loses ranking quality as the synthetic fraud becomes harder while GraphSAGE retains a positive signal, and the joint edge-head training improves the 2-layer GNN at medium/high hardness versus the previous node-only benchmark (0.671 vs 0.638 medium; 0.665 vs 0.630 high).

Experiment: message-passing depth (third layer + Jumping Knowledge)

Hypothesis from the roadmap: a 2-hop receptive field cannot distinguish a 6-cycle ring from a chain, so a deeper stack with Jumping Knowledge (concatenating every layer's embeddings) should recover longer rings. All models now support --num-layers with JK (cat/max), and the same benchmark was run at 2 and 3 layers (see the table above).

Verdict: the third layer does not help at medium/high hardness. AUC fell −0.061 (medium) and −0.035 (high) at 3 layers, with only a small gain at low hardness (+0.010). On this data the rings are short enough that a 2-hop neighborhood already contains the full ring; the extra hop aggregates mostly unrelated normal accounts and dilutes the signal. The default remains 2 layers, the depth is fully configurable, and the negative result is reported rather than tuned away.

Experiment: cycle-count features (triangle counts + clustering)

Hypothesis from the roadmap: a 2-layer GNN cannot distinguish a 6-cycle ring from a chain (both look like degree-2 neighborhoods), so explicit cycle counts might add signal. The experiment ran the same benchmark with --cycle-counts (10k accounts, 50 rings, 10 held out; full results in bench-cc/results/benchmark.json):

Hardness	Model	AUC without	AUC with	Δ
low	GraphSAGE	0.671	0.662	−0.009
low	Random Forest	0.607	0.602	−0.005
medium	GraphSAGE	0.638	0.620	−0.018
medium	Random Forest	0.520	0.526	+0.006
high	GraphSAGE	0.630	0.614	−0.016
high	Random Forest	0.495	0.537	+0.042

Verdict: the features did not help the GNN (AUC fell at every hardness), and helped Random Forest only marginally at high hardness. Ring structure on this data contains triangles (32% of ring nodes vs 24% of normal nodes), but the overlap is large enough that triangle counts mostly add noise for a model that already aggregates neighborhoods. The flag remains available (--cycle-counts) and the implementation is tested; the negative result is reported rather than tuned away. A more promising structural direction is outlined in issue #3 (transaction-level labels).

Production hardening: calibration, cold-start, and drift

Three mechanisms make the served scores more defensible in an operational setting.

Isotonic calibration. GNN raw sigmoid outputs on this data are poorly calibrated (validation brier ~0.25). train-gnn fits an isotonic regression on validation probabilities and applies it to every served score; the validation brier before/after is recorded in <model>_args.json (e.g. 0.25 → 0.02 on the demo run). The risk_score in API responses is the calibrated probability.

Cold-start fallback. Accounts with very few transactions (combined degree below --cold-start-threshold, default 10) have too little neighborhood structure for a GNN to score meaningfully. For those accounts the API routes to a class-balanced gradient-boosted tabular model trained on balance, risk_score, and age_days only, so a brand-new account still gets a principled score instead of a default. The cold-start model's training AUC and feature list are stored in the training metadata.

PSI drift monitoring. The model is trained on one graph and served against data that will drift. GET /api/drift computes the population stability index between the train-mask and test-mask calibrated score distributions (smoothed, default 10 bins): stable below 0.1, minor_drift below 0.25, and major_drift above. On the demo run the train/test split of a single generated graph reads as stable (PSI ≈ 0.004).

Transaction-level risk scores

Beyond per-account risk, the same backbone carries a transaction head: an MLP over concat(embedding[src], embedding[dst], edge_features) predicts whether a specific transfer is part of a laundering path. Edge features are log amount, hour-of-day (sin/cos), and log hours since the sender account was created. Labels are the fraud-CSV flagged transactions that sit inside a ring, and train/val/test edge masks are derived from their endpoints (an edge trains only when both endpoints train), so held-out rings never leak transactions.

The two losses are summed during training with --edge-loss-weight (default 0.5). Evaluation reports edge AUC/AP/brier alongside the node metrics, and the ring view of the dashboard exposes every internal transaction with its risk score — the graph on the right colors suspicious transfers red instead of treating a ring as one opaque blob.

Honest caveat: on the synthetic generator, flagged transactions carry a distinctive amount (₹9,999), so the edge head reaches AUC ≈ 1.0 trivially. The meaningful signal is the integration: the same embeddings drive account and transaction risk, and the edge metrics give investigators a ranked list of transfers to examine. On real data the amount encoding would be far less distinctive and the structural (embedding) side would matter more.

Risk API

upifraud serve loads a trained GNN checkpoint and graph.pt from the same output directory. It exposes both machine-readable risk scores and the dashboard endpoints.

Method	Endpoint	Purpose
GET	/healthz	Service status, model name, calibration, and node count
GET	/risk/account/{account_id}	Risk score for one account (cold-start fallback for low-activity accounts)
POST	/risk/batch	Risk scores for multiple account IDs
GET	/api/summary	Dataset and model summary (incl. calibration and cold-start status)
GET	/api/top?k=50	Highest-risk accounts
GET	/api/account/{account_id}	Account details and high-risk neighbors
GET	/api/ring/{ring_id}	Ring members, internal edges, and per-transaction risk scores
GET	/api/distribution?bins=20	Risk-score histogram
GET	/api/drift?bins=10	PSI between train and test score distributions (bins 4..50)
GET	/api/explain/{account_id}	Plain-language risk explanation + model-grounded evidence (GNNExplainer drivers)

Risk scores are in [0, 1] and mapped to bands as follows:

• low: score < 0.4;
• medium: 0.4 ≤ score < 0.7; and
• high: score ≥ 0.7.

Explanations are generated locally by default and are model-grounded: the service runs GNNExplainer on the account's 2-hop neighborhood (capped at 512 nodes for dashboard latency) and reports the features with the highest attribution masks and the most influential neighbors (model_evidence in the response). Set OPENAI_API_KEY before starting the service to enable the optional remote explanation path, which rewrites the evidence into a plain-language narrative; the service falls back to the local explanation if the request fails.

Repository layout

src/upifraud/
├── api.py          FastAPI risk service and dashboard endpoints
├── baseline.py     Random Forest, HGB, and XGBoost baselines
├── cli.py          upifraud command-line interface
├── dataset.py      CSV loading and ring-aware splitting
├── evaluate.py     AUC, AP, and ring-recovery metrics
├── features.py     Account and graph feature construction
├── generate.py     External and toy graph generators
├── models.py       GCN, GraphSAGE, and GATv2 (configurable depth/JK + edge head)
└── train.py        GNN training and checkpoint serialization

frontend/           Static dashboard assets
tests/              Unit and API tests
results/            Committed benchmark output
pyproject.toml      Package metadata and dependencies
CONTRIBUTING.md     Contribution workflow

Development

Run the test suite and linter before submitting changes:

pytest
ruff check .

For quick iteration, use the toy generator:

upifraud demo --toy --toy-accounts 120 --toy-tx 500 --rings 3

The project is intentionally synthetic and privacy-preserving. Do not add secrets, real payment data, or personally identifiable information to the repository. See CONTRIBUTING.md for the contribution workflow.

Limitations and next steps

• Synthetic data: benchmark behavior may not transfer to production payment networks.
• Node- and transaction-level labels: the current targets are ring membership and flagged transfers; richer labels (per-transaction laundering stages) are not modeled yet.
• Receptive field is configurable, not unlimited: the GNNs default to three message-passing layers with Jumping Knowledge; very long or indirect rings beyond ~3 hops still fall outside the receptive field.
• Cold-start is handled by a fallback: accounts with little graph history (combined degree below --cold-start-threshold, default 10) are scored by a class-balanced gradient-boosted tabular model instead of the GNN.
• Monitoring is partial: the service reports PSI drift between train and test score distributions (/api/drift) and a calibrated operating point, but investigator feedback loops are not implemented.
• Unused text/embeddings: generated descriptions and embeddings are not yet consumed by the model.

Planned directions include temporal (dynamic-graph) modeling, adversarial-robustness tests, counterfactual explanations, and scaling the benchmark to larger graphs.

License

Mule-Hunt is released under the MIT License. The synthetic graph generator is a separate dependency distributed under Apache-2.0.