Metadata-Version: 2.4
Name: mule-hunt
Version: 0.3.0
Summary: Graph-neural-network fraud detection for UPI-style payment graphs
Author: Soham Jadhav
License: MIT
Keywords: gnn,fraud-detection,upi,graph-neural-networks,aml
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: torch>=2.2
Requires-Dist: torch-geometric>=2.5
Requires-Dist: scikit-learn>=1.3
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn>=0.29
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Provides-Extra: xgb
Requires-Dist: xgboost>=2.0; extra == "xgb"
Dynamic: license-file

# Mule-Hunt

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)

Mule-Hunt is a graph-neural-network (GNN) pipeline for detecting coordinated
fraud in UPI-style payment networks.

The project represents payment activity as a graph:

- an account is a node;
- a transfer is a directed edge; and
- a planted fraud ring is a group of accounts that move money in a cycle.

The model predicts which accounts belong to a fraud ring. This lets the
pipeline evaluate network-level signals—such as unusual connectivity and
coordinated cycles—that are invisible when each transaction is considered in
isolation.

> **Research/demo project:** all experiments use synthetic data. The scores in
> this repository are not production fraud-detection claims and must not be
> used to make decisions about real accounts.

## What is included

- Synthetic UPI-style graph generation using
  [SantanderAI/gen-fraud-graph](https://github.com/SantanderAI/gen-fraud-graph).
- A small built-in generator for tests and offline smoke runs.
- CSV-to-[PyTorch Geometric](https://pyg.org/) graph loading.
- Ring-aware train/validation/test splits that hold out complete fraud rings.
- Three GNNs: GCN, GraphSAGE, and GATv2 — with configurable depth
  (message-passing layers) and Jumping-Knowledge aggregation instead of a
  hard-coded 2-hop receptive field.
- A transaction-level head on the same backbone: per-transfer risk scores from
  node embeddings + amount/time features, trained jointly with the node loss.
- Non-graph baselines: Random Forest, HistGradientBoosting, and optional
  XGBoost.
- Leak-aware node feature construction and class-imbalance handling.
- Isotonic probability calibration with validation-brier reporting.
- A cold-start fallback (gradient-boosted tabular model) for low-activity
  accounts, whose neighborhoods are too small for a GNN to score.
- Population-stability-index (PSI) drift monitoring between train and test
  score distributions.
- AUC, average precision, and whole-ring recovery metrics.
- A FastAPI risk-scoring service with a lightweight dashboard.
- Optional plain-language account explanations, with a deterministic local
  fallback when no API key is configured.

## Dashboard

![Mule-Hunt risk dashboard](docs/dashboard.png)

`upifraud serve` runs the FastAPI backend plus a browser dashboard: summary
metrics, the top-50 highest-risk accounts, a risk-score histogram, ring
exploration on a live graph, per-account neighborhoods, and plain-English risk
explanations (optionally generated by an LLM via `OPENAI_API_KEY`, with a
deterministic local fallback).

Explanations are model-grounded: [GNNExplainer](https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.explain.algorithm.GNNExplainer.html)
runs on the account's 2-hop neighborhood and reports which features and which
neighbors drive the risk score:

![Model-grounded explanation](docs/explain.png)

## How the pipeline works

```text
synthetic CSV graph
        │
        ▼
account and transaction tables
        │
        ▼
PyG Data object
  ├── node features: account + connectivity
  ├── directed transaction edges
  ├── fraud-ring labels
  └── ring-aware train/validation/test masks
        │
        ├── GraphSAGE / GCN
        └── tabular baselines
                │
                ▼
       held-out evaluation + risk scores
                │
                ▼
        FastAPI service + dashboard
```

## Requirements

- Python 3.11 or newer
- Git, because the synthetic graph generator is installed from GitHub
- `uv` is recommended for environment and dependency management
- A CPU is sufficient for the included demo and tests

## Installation

Using `uv`:

```bash
git clone https://github.com/sohamvjadhav/Mule-Hunt.git
cd Mule-Hunt

uv venv --python 3.12
source .venv/bin/activate       # Windows PowerShell: .venv\Scripts\Activate.ps1
uv pip install -e ".[dev]"
```

Without `uv`:

```bash
python3 -m venv .venv
source .venv/bin/activate       # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
```

Verify the installation:

```bash
upifraud --version
pytest
```

## Quick start

Run the complete pipeline on a small synthetic graph:

```bash
upifraud demo --toy --rings 5
```

This command generates a toy graph, trains GraphSAGE, trains Random Forest and
HistGradientBoosting baselines, evaluates all saved models, and prints the
highest-scoring accounts. It writes generated data to `data/raw` and model
artifacts to `models`.

For a small graph produced by the external generator instead of the built-in
toy generator:

```bash
upifraud demo --data data/external --out-dir models-external \
  --scale 0.0001 --rings 10
```

After training, start the API and dashboard:

```bash
upifraud serve --out-dir models
```

Open <http://127.0.0.1:8000> in a browser. The API is also available at:

```bash
curl http://127.0.0.1:8000/healthz
curl http://127.0.0.1:8000/risk/account/acc_42

curl -X POST http://127.0.0.1:8000/risk/batch \
  -H 'Content-Type: application/json' \
  -d '{"account_ids": ["acc_42", "acc_99"]}'
```

The account IDs depend on the generated dataset, so replace `acc_42` and
`acc_99` with IDs that exist in your graph.

## Command-line reference

The package exposes one command, `upifraud`, with the following subcommands.

### Generate data

```bash
upifraud generate --output data/raw --scale 0.001 --rings 50
```

Use the built-in generator for a fast, deterministic smoke run:

```bash
upifraud generate --toy --output data/toy --toy-accounts 300 \
  --toy-tx 2500 --rings 5 --seed 42
```

Important generation options:

| Option | Default | Description |
| --- | ---: | --- |
| `--output` | `data/raw` | Directory for graph CSVs |
| `--scale` | `0.001` | Scale passed to `gen-fraud-graph` |
| `--rings` | generator default | Number of planted fraud rings |
| `--hardness` | `low` | Synthetic difficulty: `low`, `medium`, or `high` |
| `--workers` | `1` | Generator worker count |
| `--toy` | off | Use the built-in generator |
| `--seed` | `42` | Seed for the toy generator |

The generated directory contains `accounts/`, `transactions/`, and `fraud/`.

### Train a GNN

```bash
upifraud train-gnn \
  --data data/raw \
  --out-dir models \
  --model sage \
  --epochs 200 \
  --test-rings 3
```

Available GNNs are `sage`, `gcn`, and `gat`. The default `rings` split holds out
whole rings for testing. Use `--split random` only when you specifically want a
random node split for comparison; it is less representative of discovering a
previously unseen ring.

All three architectures are configurable in depth: `--num-layers` (default 2)
stacks message-passing layers and Jumping Knowledge (`--jk cat`, the default,
or `max`) concatenates every layer's embeddings before the classifier, so the
receptive field is 1..N hops and deeper stacks avoid over-smoothing. Use
`--edge-loss-weight 0` to drop the transaction-level head and train node-only
(the edge head is trained jointly by default with weight 0.5).

The command saves:

- `<model>.pt`: model weights;
- `<model>_args.json`: model dimensions, feature standardization values,
  validation brier before/after calibration, and cold-start metadata;
- `<model>_calib.pkl`: isotonic calibrator fitted on validation scores;
- `<model>_coldstart.joblib`: cold-start tabular model and metadata; and
- `graph.pt`: the processed graph and its split masks.

Calibration and cold-start are on by default and can be adjusted:

| Option | Default | Description |
| --- | ---: | --- |
| `--calibrate` / `--no-calibrate` | on | Fit an isotonic calibrator on validation scores |
| `--cold-start-threshold` | `10` | Combined degree below which a tabular cold-start score is used instead of the GNN |
| `--num-layers` | `2` | Message-passing depth; Jumping Knowledge aggregates all layers |
| `--jk` | `cat` | JK aggregation: `cat` or `max` |
| `--edge-loss-weight` | `0.5` | Weight of the transaction-level loss (set to `0` to disable the edge head) |

### Train a baseline

```bash
upifraud train-baseline \
  --data data/raw \
  --out-dir models \
  --model rf
```

Available baselines are `rf`, `hgb`, and `xgb`. Install the optional XGBoost
dependency before using `xgb`:

```bash
python -m pip install -e ".[dev,xgb]"
```

### Compare saved models

```bash
upifraud evaluate --out-dir models
```

This reads the saved graph and model artifacts and prints AUC, average
precision, mean ring recall, and fraud hit rate at the evaluation cutoff.

### Run the benchmark matrix

```bash
upifraud benchmark \
  --root bench \
  --scale 0.001 \
  --rings 50 \
  --test-rings 10
```

The benchmark trains the selected GNN and baselines at each requested
hardness level and writes `bench/results/benchmark.json`. Add
`--regenerate` to replace already-generated benchmark data.

## Data and features

The external generator is the open-source synthetic
[gen-fraud-graph](https://github.com/SantanderAI/gen-fraud-graph) project,
used under Apache-2.0. No real financial data is included or required.

Mule-Hunt loads account CSVs, transaction CSVs, fraud transaction labels, and
fraud-case metadata into one PyG graph. Each account receives a node label of
`1` when it belongs to a planted ring and `0` otherwise.

By default, the node representation uses account and structural features:

- log balance;
- account risk score;
- account age;
- in-degree and out-degree; and
- unique inbound and outbound counterparties.

Constant columns are removed before training. `--amount-stats` adds inbound and
outbound amount aggregates, but those features are intentionally excluded by
default: the synthetic generator uses a distinctive amount for planted ring
transactions, so amount aggregates can reveal the label through a benchmark
artifact rather than through network structure.

`--cycle-counts` adds per-node triangle counts and the local clustering
coefficient (undirected 3-cycle structure, computed in milliseconds on the
10k-node graph via sparse adjacency multiplication). The experiment is
documented below: on gen-fraud-graph data, 32% of ring nodes sit in at least
one triangle (vs 24% of normal nodes), but the overlap is large and the
features did not help the GNN.

## Evaluation protocol

The default evaluation is designed to test whether a model can find new rings:

1. Fraud rings are split as groups, not as individual accounts.
2. All members of a held-out ring stay in the same test split.
3. Normal accounts are sampled into train, validation, and test splits.
4. Training uses class-weighted binary cross-entropy and early stopping on
   validation average precision.
5. Results are reported on the held-out test accounts.

The main metrics are:

- **AUC:** ranking quality across positive and negative accounts;
- **average precision (AP):** more informative than accuracy for the imbalanced
  fraud labels;
- **mean ring recall:** the average fraction of each held-out ring recovered in
  the top-`k` ranked accounts, where `k` is the test-set size; and
- **operating point:** precision/recall at the F1-maximizing score threshold on
  held-out rings — the number an investigator would actually deploy with.

### Current benchmark results

The committed benchmark uses approximately 10,000 accounts, 90,000 transfers,
50 rings, and 10 held-out rings. Runs include the transaction-level head
(edge AUC is reported on held-out transactions). The generator has its own
randomness, so numbers may vary slightly between runs.

| Hardness | Model | Layers | AUC | AP | Mean ring recall | Edge AUC |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| low | GraphSAGE | 2 | 0.620 | 0.056 | 0.238 | 1.000 |
| low | GraphSAGE | 3 | **0.630** | 0.058 | **0.273** | 1.000 |
| low | Random Forest | – | 0.507 | 0.036 | 0.142 | – |
| medium | GraphSAGE | 2 | **0.671** | **0.079** | **0.359** | 0.999 |
| medium | GraphSAGE | 3 | 0.610 | 0.051 | 0.324 | 0.999 |
| medium | Random Forest | – | 0.603 | 0.065 | 0.269 | – |
| high | GraphSAGE | 2 | **0.665** | 0.069 | **0.351** | 0.997 |
| high | GraphSAGE | 3 | 0.629 | 0.044 | 0.262 | 0.997 |
| high | Random Forest | – | 0.622 | 0.057 | 0.185 | – |

Full results are in [`results/benchmark.json`](results/benchmark.json). The
important comparisons: Random Forest loses ranking quality as the synthetic
fraud becomes harder while GraphSAGE retains a positive signal, and the joint
edge-head training improves the 2-layer GNN at medium/high hardness versus the
previous node-only benchmark (0.671 vs 0.638 medium; 0.665 vs 0.630 high).

### Experiment: message-passing depth (third layer + Jumping Knowledge)

Hypothesis from the roadmap: a 2-hop receptive field cannot distinguish a
6-cycle ring from a chain, so a deeper stack with Jumping Knowledge
(concatenating every layer's embeddings) should recover longer rings. All
models now support `--num-layers` with JK (`cat`/`max`), and the same
benchmark was run at 2 and 3 layers (see the table above).

**Verdict: the third layer does not help at medium/high hardness.** AUC fell
−0.061 (medium) and −0.035 (high) at 3 layers, with only a small gain at low
hardness (+0.010). On this data the rings are short enough that a 2-hop
neighborhood already contains the full ring; the extra hop aggregates mostly
unrelated normal accounts and dilutes the signal. The default remains 2
layers, the depth is fully configurable, and the negative result is reported
rather than tuned away.

### Experiment: cycle-count features (triangle counts + clustering)

Hypothesis from the roadmap: a 2-layer GNN cannot distinguish a 6-cycle ring
from a chain (both look like degree-2 neighborhoods), so explicit cycle counts
might add signal. The experiment ran the same benchmark with `--cycle-counts`
(10k accounts, 50 rings, 10 held out; full results in
`bench-cc/results/benchmark.json`):

| Hardness | Model | AUC without | AUC with | Δ |
| --- | --- | ---: | ---: | ---: |
| low | GraphSAGE | 0.671 | 0.662 | −0.009 |
| low | Random Forest | 0.607 | 0.602 | −0.005 |
| medium | GraphSAGE | 0.638 | 0.620 | −0.018 |
| medium | Random Forest | 0.520 | 0.526 | +0.006 |
| high | GraphSAGE | 0.630 | 0.614 | −0.016 |
| high | Random Forest | 0.495 | 0.537 | +0.042 |

**Verdict: the features did not help the GNN** (AUC fell at every hardness),
and helped Random Forest only marginally at high hardness. Ring structure on
this data contains triangles (32% of ring nodes vs 24% of normal nodes), but
the overlap is large enough that triangle counts mostly add noise for a model
that already aggregates neighborhoods. The flag remains available
(`--cycle-counts`) and the implementation is tested; the negative result is
reported rather than tuned away. A more promising structural direction is
outlined in [issue #3](https://github.com/sohamvjadhav/Mule-Hunt/issues/3)
(transaction-level labels).

## Production hardening: calibration, cold-start, and drift

Three mechanisms make the served scores more defensible in an operational
setting.

**Isotonic calibration.** GNN raw sigmoid outputs on this data are poorly
calibrated (validation brier ~0.25). `train-gnn` fits an
[isotonic regression](https://scikit-learn.org/stable/modules/generated/sklearn.isotonic.IsotonicRegression.html)
on validation probabilities and applies it to every served score; the
validation brier before/after is recorded in `<model>_args.json` (e.g.
0.25 → 0.02 on the demo run). The `risk_score` in API responses is the
calibrated probability.

**Cold-start fallback.** Accounts with very few transactions (combined degree
below `--cold-start-threshold`, default 10) have too little neighborhood
structure for a GNN to score meaningfully. For those accounts the API routes to
a class-balanced gradient-boosted tabular model trained on
`balance`, `risk_score`, and `age_days` only, so a brand-new account still gets
a principled score instead of a default. The cold-start model's training AUC
and feature list are stored in the training metadata.

**PSI drift monitoring.** The model is trained on one graph and served against
data that will drift. `GET /api/drift` computes the
[population stability index](https://www.lexjansen.com/nesug/nesug06/cc/cc07.pdf)
between the train-mask and test-mask calibrated score distributions (smoothed,
default 10 bins): `stable` below 0.1, `minor_drift` below 0.25, and
`major_drift` above. On the demo run the train/test split of a single generated
graph reads as stable (PSI ≈ 0.004).

## Transaction-level risk scores

Beyond per-account risk, the same backbone carries a transaction head: an MLP
over `concat(embedding[src], embedding[dst], edge_features)` predicts whether a
specific transfer is part of a laundering path. Edge features are log amount,
hour-of-day (sin/cos), and log hours since the sender account was created.
Labels are the fraud-CSV flagged transactions that sit inside a ring, and
train/val/test edge masks are derived from their endpoints (an edge trains
only when both endpoints train), so held-out rings never leak transactions.

The two losses are summed during training with `--edge-loss-weight` (default
0.5). Evaluation reports edge AUC/AP/brier alongside the node metrics, and the
ring view of the dashboard exposes every internal transaction with its risk
score — the graph on the right colors suspicious transfers red instead of
treating a ring as one opaque blob.

> **Honest caveat:** on the synthetic generator, flagged transactions carry a
> distinctive amount (₹9,999), so the edge head reaches AUC ≈ 1.0 trivially.
> The meaningful signal is the *integration*: the same embeddings drive account
> and transaction risk, and the edge metrics give investigators a ranked list
> of transfers to examine. On real data the amount encoding would be far less
> distinctive and the structural (embedding) side would matter more.

## Risk API

`upifraud serve` loads a trained GNN checkpoint and `graph.pt` from the same
output directory. It exposes both machine-readable risk scores and the
dashboard endpoints.

| Method | Endpoint | Purpose |
| --- | --- | --- |
| `GET` | `/healthz` | Service status, model name, calibration, and node count |
| `GET` | `/risk/account/{account_id}` | Risk score for one account (cold-start fallback for low-activity accounts) |
| `POST` | `/risk/batch` | Risk scores for multiple account IDs |
| `GET` | `/api/summary` | Dataset and model summary (incl. calibration and cold-start status) |
| `GET` | `/api/top?k=50` | Highest-risk accounts |
| `GET` | `/api/account/{account_id}` | Account details and high-risk neighbors |
| `GET` | `/api/ring/{ring_id}` | Ring members, internal edges, and per-transaction risk scores |
| `GET` | `/api/distribution?bins=20` | Risk-score histogram |
| `GET` | `/api/drift?bins=10` | PSI between train and test score distributions (bins 4..50) |
| `GET` | `/api/explain/{account_id}` | Plain-language risk explanation + model-grounded evidence (GNNExplainer drivers) |

Risk scores are in `[0, 1]` and mapped to bands as follows:

- `low`: score `< 0.4`;
- `medium`: `0.4 ≤ score < 0.7`; and
- `high`: score `≥ 0.7`.

Explanations are generated locally by default and are model-grounded: the
service runs [GNNExplainer](https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.explain.algorithm.GNNExplainer.html)
on the account's 2-hop neighborhood (capped at 512 nodes for dashboard
latency) and reports the features with the highest attribution masks and the
most influential neighbors (`model_evidence` in the response). Set
`OPENAI_API_KEY` before starting the service to enable the optional remote
explanation path, which rewrites the evidence into a plain-language narrative;
the service falls back to the local explanation if the request fails.

## Repository layout

```text
src/upifraud/
├── api.py          FastAPI risk service and dashboard endpoints
├── baseline.py     Random Forest, HGB, and XGBoost baselines
├── cli.py          upifraud command-line interface
├── dataset.py      CSV loading and ring-aware splitting
├── evaluate.py     AUC, AP, and ring-recovery metrics
├── features.py     Account and graph feature construction
├── generate.py     External and toy graph generators
├── models.py       GCN, GraphSAGE, and GATv2 (configurable depth/JK + edge head)
└── train.py        GNN training and checkpoint serialization

frontend/           Static dashboard assets
tests/              Unit and API tests
results/            Committed benchmark output
pyproject.toml      Package metadata and dependencies
CONTRIBUTING.md     Contribution workflow
```

## Development

Run the test suite and linter before submitting changes:

```bash
pytest
ruff check .
```

For quick iteration, use the toy generator:

```bash
upifraud demo --toy --toy-accounts 120 --toy-tx 500 --rings 3
```

The project is intentionally synthetic and privacy-preserving. Do not add
secrets, real payment data, or personally identifiable information to the
repository. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the contribution
workflow.

## Limitations and next steps

- **Synthetic data:** benchmark behavior may not transfer to production payment
  networks.
- **Node- and transaction-level labels:** the current targets are ring
  membership and flagged transfers; richer labels (per-transaction laundering
  stages) are not modeled yet.
- **Receptive field is configurable, not unlimited:** the GNNs default to
  three message-passing layers with Jumping Knowledge; very long or indirect
  rings beyond ~3 hops still fall outside the receptive field.
- **Cold-start is handled by a fallback:** accounts with little graph history
  (combined degree below `--cold-start-threshold`, default 10) are scored by a
  class-balanced gradient-boosted tabular model instead of the GNN.
- **Monitoring is partial:** the service reports PSI drift between train and
  test score distributions (`/api/drift`) and a calibrated operating point, but
  investigator feedback loops are not implemented.
- **Unused text/embeddings:** generated descriptions and embeddings are not yet
  consumed by the model.

Planned directions include temporal (dynamic-graph) modeling, adversarial-robustness
tests, counterfactual explanations, and scaling the benchmark to larger graphs.

## License

Mule-Hunt is released under the [MIT License](LICENSE). The synthetic graph
generator is a separate dependency distributed under Apache-2.0.
