Metadata-Version: 2.4
Name: rpdcnn
Version: 0.1.0
Summary: RPDCNN: a multi-task perception model for detection and polygon-aware segmentation
Author: Pavan Teja
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Requires-Dist: opencv-python-headless>=4.8
Requires-Dist: timm>=1.0.0
Requires-Dist: torch>=2.1
Requires-Dist: torchvision>=0.16

# RPDCNN

**RPDCNN** (Region Proposal / Dynamic Convolution Neural Network) is a single-stage, anchor-free, multi-task perception model that performs **object detection + instance segmentation** on a single forward pass, and emits **polygon-shaped masks** (YOLO-seg format) rather than raw pixel blobs.

It's built as an FCOS-style detector with a CondInst-style dynamic ("controller-generated") mask head — the wiring intentionally mirrors [AdelaiDet's](https://github.com/aim-uofa/AdelaiDet) `CondInst` implementation — with a BiFPN neck and optional CBAM attention bolted on for extra feature quality.

> No description, license, or topics are set on the upstream GitHub repo at the time of writing. This README was generated by reading the source directly.

---

## Table of contents

- [Why RPDCNN](#why-rpdcnn)
- [Architecture](#architecture)
- [Repository layout](#repository-layout)
- [Installation](#installation)
- [Quickstart](#quickstart)
- [Loading a model from a checkpoint](#4-load-a-trained-model-from-a-checkpoint)
- [Label format](#label-format-yolo-seg)
- [Configuration](#configuration)
- [Component reference](#component-reference)
- [Losses](#losses)
- [Checkpointing & resuming](#checkpointing--resuming)
- [Output format](#output-format)
- [References](#references)

---

## Why RPDCNN

Most instance-segmentation stacks (Mask R-CNN, etc.) are two-stage: propose boxes, then crop-and-segment. RPDCNN instead:

1. Regresses boxes **and** generates a small set of per-instance "dynamic" convolution weights at every FCOS point location, in one shot (no RoIAlign, no proposal stage).
2. Applies those per-instance weights to a shared, high-resolution mask-feature map to produce a dense mask for each detected instance (the CondInst trick).
3. Converts every dense mask to a polygon (`cv2.findContours` + `approxPolyDP`) at inference time, so the model's public output is directly usable as YOLO-seg labels.

Training only ever supervises dense bitmasks (dice loss) — the polygon step is a post-processing convenience on the output side, not part of the training loop.

---

## Architecture

```
                                   ┌───────────────────────────┐
                                   │          IMAGE            │
                                   │   (B, 3, img_h, img_w)     │
                                   └─────────────┬─────────────┘
                                                 │
                                                 ▼
                                   ┌───────────────────────────┐
                                   │         BACKBONE          │
                                   │   timm, features_only     │
                                   │ convnext_small / resnet50 │
                                   │ efficientnet_b3 / mnetv3  │
                                   │      / regnety_008        │
                                   └─────────────┬─────────────┘
                                                 │  C_i @ in_strides (e.g. [4,8,16,32])
                                                 ▼
                                   ┌───────────────────────────┐
                                   │           NECK            │
                                   │   BiFPN (learned fusion)  │
                                   │      or torchvision FPN   │
                                   │        (+P6/P7 extras)    │
                                   └─────────────┬─────────────┘
                                                 │  P_i  (named "p{log2(stride)}")
                                                 ▼
                                   ┌───────────────────────────┐
                                   │      CBAM  (optional)     │
                                   │  channel-attn + spatial-  │
                                   │   attn, applied per level │
                                   └─────────────┬─────────────┘
                                                 │
                          ┌──────────────────────┴───────────────────────┐
                          ▼                                              ▼
           ┌───────────────────────────┐                  ┌───────────────────────────┐
           │     DynamicDetHead         │                 │        MaskBranch          │
           │   (applied per level)      │                 │  fuses a subset of levels  │
           │                            │                 │  (e.g. p3,p4,p5[,p6])      │
           │  cls_tower  → cls_logits   │                 │  refine + top-down add     │
           │  bbox_tower → bbox_preds   │                 │  + conv tower              │
           │            → centerness   │                 │      → mask_feats          │
           │            → controller   │                 │  (shared feature map,      │
           │        (dynamic weights)  │                 │   stride = mask_branch_    │
           └─────────────┬─────────────┘                 │        out_stride)         │
                         │                                └─────────────┬─────────────┘
                         │                                              │
             (training)  ▼                                              │
           ┌───────────────────────────┐                                │
           │       FCOSAssigner        │                                │
           │  center-sampling + size-  │                                │
           │  of-interest range match  │                                │
           │   GT box/mask → point     │                                │
           └─────────────┬─────────────┘                                │
                         │ matched_gt_inds (positives)                  │
                         ▼                                              │
           ┌───────────────────────────┐                                │
           │  gather positive points   │                                │
           │  across all levels/images │                                │
           │  → controller params      │◄───────────────────────────────┘
           └─────────────┬─────────────┘
                         │
                         ▼
           ┌───────────────────────────┐
           │      DynamicMaskHead       │
           │  per-instance controller  │
           │  params → conv weights,   │
           │  applied over mask_feats  │
           │  (+ relative coords)      │
           │      → mask_logits        │
           │   (dense, per instance)   │
           └─────────────┬─────────────┘
                         │
        ┌────────────────┴─────────────────┐
        ▼ training                          ▼ inference
 ┌─────────────────┐            ┌─────────────────────────────┐
 │  Dice + BCE loss │            │  sigmoid → threshold        │
 │  vs GT bitmasks  │            │  cv2.findContours +         │
 │  (strided from   │            │  approxPolyDP               │
 │  full-res raster)│            │  → normalized YOLO polygon  │
 └─────────────────┘            └─────────────────────────────┘
```

**Data flow summary:** `image → Backbone → Neck (BiFPN/FPN) → CBAM → { DynamicDetHead (per level) , MaskBranch (fused levels) } → FCOSAssigner (train) → gather positives → DynamicMaskHead → dense masks → (inference only) polygons`.

---

## Repository layout

```
RPDCNN/
├── README.md
├── pyproject.toml
├── requirements.txt
└── src/
    └── rpdcnn/
        ├── __init__.py              # public API: RPDCNN, RPDCFG, get_default_cfg, get_preset_cfg
        ├── cfg.py                   # RPDCFG dataclass + "nano" / "fast" / "main" presets
        ├── rpdcnn.py                # top-level nn.Module: forward(), train_model(), predict(), infer()
        ├── feature_extractor/
        │   ├── backbone.py          # timm wrapper (features_only=True)
        │   ├── bifpn.py             # weighted bidirectional FPN
        │   ├── fpn.py               # torchvision FeaturePyramidNetwork wrapper (+P6/P7)
        │   └── cbam.py              # channel + spatial attention block
        ├── heads/
        │   ├── dynamic_det_head.py  # FCOS-style cls/box/centerness + controller head
        │   ├── mask_branch.py       # fuses neck levels into a shared mask feature map
        │   └── dynamic_mask_head.py # applies per-instance dynamic conv weights
        └── utils/
            ├── fcos_assigner.py     # GT-to-point assignment (center sampling, size-of-interest ranges)
            ├── losses.py            # IOULoss (GIoU/IoU), focal loss, compute_fcos_losses
            ├── mask_utils.py        # aligned_bilinear, dice_coefficient, parse_dynamic_params, etc.
            ├── yolo_poly_dataset.py # YOLO-seg dataset + collate_fn
            └── viz_utils.py         # per-epoch GT-vs-prediction visualization panels
```

---

## Installation

Requires **Python ≥ 3.10**.

```bash
git clone https://github.com/iampa1teja/RPDCNN.git
cd RPDCNN
pip install -e .
```

or, without an editable install:

```bash
pip install -r requirements.txt
```

Dependencies (from `pyproject.toml` / `requirements.txt`):

| Package | Version |
|---|---|
| `numpy` | `>=1.24` |
| `opencv-python-headless` | `>=4.8` |
| `timm` | `>=1.0.0` |
| `torch` | `>=2.1` |
| `torchvision` | `>=0.16` |

---

## Quickstart

### 1. Build a model from a preset

```python
from rpdcnn import get_preset_cfg, RPDCNN

# presets: "nano" (mobilenetv3, fast/light), "fast" (resnet50), "main" (convnext_small, full accuracy)
cfg = get_preset_cfg("main", num_classes=3)
model = RPDCNN(cfg)
```

Or start from defaults and override fields manually:

```python
from rpdcnn import get_default_cfg, RPDCNN

cfg = get_default_cfg()
cfg.num_classes = 5
cfg.img_h = cfg.img_w = 640
model = RPDCNN(cfg)
```

### 2. Train on a YOLO-segmentation dataset

```python
model.train_model(
    img_dir="data/images",
    label_dir="data/labels",
    format="yolo_poly",              # only supported format right now
    output_dir="./rpdcnn_output",
    num_epochs=50,
    batch_size=4,
    lr=1e-4,
    weight_decay=1e-4,
    num_workers=4,
    save_freq=5,
    class_names=["beaker", "test_tube", "others"],
)
```

Every epoch this:
- runs one full pass over the dataset and backprops the summed loss dict,
- writes a GT-vs-prediction visualization panel to `output_dir/viz/`.

Every `save_freq` epochs (and at the final epoch) it saves a checkpoint to `output_dir/checkpoints/epoch_XXXX.pt` (model + optimizer + cfg).

To resume, set on the config before constructing/training:

```python
cfg.resume_training = True
cfg.resume_checkpoint_path = ""   # empty -> auto-picks latest checkpoint under output_dir/checkpoints/
```

### 3. Run inference on a single image

```python
result, viz = model.predict(
    model,
    image_path="sample.jpg",
    score_thresh=0.3,
    nms_iou_threshold=0.5,
    mask_threshold=0.5,
    approx_epsilon_frac=0.005,
)

print(result["instances"])   # [{"class_id":..., "score":..., "polygon":[[x,y],...]}, ...]
```

`result["instances"][i]["polygon"]` is a list of **absolute pixel** `(x, y)` coordinates at the image's original resolution.

### 4. Load a trained model from a checkpoint

Checkpoints saved by `train_model()` bundle `cfg` alongside the weights (see [Checkpointing & resuming](#checkpointing--resuming)), so a model can be rebuilt without redeclaring `RPDCFG` by hand. The saved `cfg` is a **plain dict** (`RPDCFG.to_dict()`), not an `RPDCFG` instance — it must be converted back with `RPDCFG.from_dict()` before it's passed into `RPDCNN(...)`, since the model reads config fields as attributes (`cfg.num_classes`, etc.), not dict keys.

```python
import torch
from rpdcnn import RPDCNN
from rpdcnn.cfg import RPDCFG

ckpt_path = "last.pt"
device = "cuda" if torch.cuda.is_available() else "cpu"

ckpt = torch.load(ckpt_path, map_location=device)

cfg = ckpt["cfg"]
if isinstance(cfg, dict):
    cfg = RPDCFG.from_dict(cfg)   # checkpoint stores cfg as a plain dict — convert back

model = RPDCNN(cfg)                              # rebuild architecture from saved cfg
model.load_state_dict(ckpt["model_state_dict"])  # load trained weights
model.to(device)
model.eval()

result, viz = model.predict(image_path="img.jpg")
print(result)
print(viz)
```

### 5. Run inference over a folder and save JSON + overlays

```python
model.infer(
    img_dir="data/test_images",
    output_dir="./rpdcnn_infer_output",
    score_thresh=0.3,
    nms_iou_threshold=0.5,
    mask_threshold=0.5,
)
```

This writes, per image:
```
rpdcnn_infer_output/
├── json/<stem>.json   # {"image_path","width","height","instances":[...]}
└── viz/<stem>.png     # original image with mask overlay + polygon outline + class legend
```

### 6. Raw forward pass (custom decode pipeline)

```python
model.eval()
out = model(images)  # images: (B, 3, img_h, img_w) tensor, already resized

out["polygons"]           # built-in decoded output
out["bbox_preds"]          # (B, N, 4) raw per-point box regressions
out["cls_logits"]          # (B, N, num_classes)
out["centerness_logits"]   # (B, N, 1)
out["controller"]          # (B, N, controller_dim)
out["mask_feats"]          # shared mask feature map
out["points_flat"], out["feature_shapes"]
```

---

## Label format (YOLO-seg)

`YoloPolyDataset` expects:

```
img_dir/
├── image_0001.jpg
└── image_0002.png
label_dir/
├── image_0001.txt
└── image_0002.txt
```

Each `.txt` has one line per instance:

```
class_id x1 y1 x2 y2 x3 y3 ... xn yn
```

All coordinates are normalized to `[0, 1]`. Each line is one ring / one instance (multi-ring instances aren't supported by the plain YOLO-seg format). Internally, polygons are rasterized **once**, at full input resolution, into a dense bitmask (`_add_bitmasks`, mirroring AdelaiDet's `CondInst.add_bitmasks`); stride-specific targets are obtained by pure strided subsampling of that full-res mask — never a separate resize.

---

## Configuration

All knobs live in the `RPDCFG` dataclass (`cfg.py`):

| Field | Default | Notes |
|---|---|---|
| `num_classes` | `80` | |
| `in_strides` | `[4, 8, 16, 32]` | backbone levels to extract |
| `img_h` / `img_w` | `512` / `512` | |
| `backbone_name` | `convnext_small` | one of `convnext_small`, `resnet50`, `efficientnet_b3`, `mobilenetv3_large_100`, `regnety_008` |
| `backbone_pretrained` | `True` | |
| `freeze_backbone` / `freeze_backbone_bn` | `False` | |
| `neck_type` | `bifpn` | `bifpn` or `fpn` (torchvision, adds P6/P7 automatically) |
| `bifpn_out_channels` / `bifpn_num_layers` | `256` / `3` | |
| `use_cbam` / `cbam_reduction` | `True` / `16` | |
| `det_num_convs` / `det_norm_groups` | `4` / `8` | detection head tower depth |
| `mask_branch_in_features` | `["p3","p4","p5"]` | which neck levels feed the mask branch |
| `mask_branch_channels` / `out_channels` / `num_convs` | `128` / `8` / `4` | |
| `mask_branch_out_stride` | `8` | stride of the shared mask feature map |
| `mask_head_num_layers` / `channels` | `3` / `8` | dynamic conv stack |
| `mask_head_out_stride` | `4` | final mask resolution stride |
| `mask_head_sizes_of_interest` | `[64,128,256]` | relative-coord normalization per FPN level |
| `assigner_center_sampling_radius` | `1.5` | FCOS center sampling |
| `assigner_regress_ranges` | `[(-1,64),(64,128),(128,100000)]` | per-level box-size assignment ranges — **must** have one entry per neck level |
| `loss_focal_alpha` / `loss_focal_gamma` | `0.25` / `2.0` | classification focal loss |
| `loss_iou_type` | `giou` | box regression loss (`iou` or `giou`) |

### Built-in presets (`get_preset_cfg(name, **overrides)`)

| Preset | Backbone | Neck | CBAM | Intended use |
|---|---|---|---|---|
| `nano` | `mobilenetv3_large_100` (frozen) | `fpn` | off | fastest / lowest memory |
| `fast` | `resnet50` | `bifpn` (192ch) | on | balanced speed/accuracy |
| `main` | `convnext_small` | `bifpn` (256ch, 5 levels) | on | full accuracy |

Overrides are validated against `RPDCFG`'s field set and applied on top of the preset (`dataclasses.replace`), so unset fields fall back to `RPDCFG`'s own defaults rather than the preset silently resetting them.

---

## Component reference

- **Backbone** (`feature_extractor/backbone.py`) — thin `timm.create_model(..., features_only=True)` wrapper; picks out only the feature levels matching `in_strides` and exposes their channel counts.
- **BiFPN** (`feature_extractor/bifpn.py`) — learned top-down + bottom-up weighted fusion (`BiFPNLayer`) over depthwise-separable convs, repeated `bifpn_num_layers` times.
- **FPN** (`feature_extractor/fpn.py`) — alternative neck using `torchvision.ops.FeaturePyramidNetwork` with `LastLevelP6P7` extras.
- **CBAM** (`feature_extractor/cbam.py`) — sequential channel-attention (avg+max pool → shared MLP) then spatial-attention (channel avg+max → 7×7 conv), applied per neck level.
- **DynamicDetHead** (`heads/dynamic_det_head.py`) — FCOS-style head: separate cls/bbox towers, per-level learnable `Scale`, outputs `(bbox_preds, cls_logits, centerness_logits, controller)`.
- **MaskBranch** (`heads/mask_branch.py`) — refines and top-down-fuses the chosen neck levels into one shared `mask_feats` tensor; optional auxiliary semantic-segmentation loss (`mask_branch_sem_loss_on`).
- **DynamicMaskHead** (`heads/dynamic_mask_head.py`) — CondInst-style controller-conv head: unpacks each instance's controller vector into conv weights/biases (`parse_dynamic_params`), runs them as grouped 1×1 convs over `mask_feats` + relative-coordinate channels, upsamples via `aligned_bilinear`.
- **FCOSAssigner** (`utils/fcos_assigner.py`) — assigns GT boxes to point locations per level using center sampling + per-level regress ranges.

---

## Losses

Returned as a dict from `forward()` in training mode and summed for backprop:

- `loss_cls` — sigmoid focal loss on classification logits.
- `loss_bbox` — IoU/GIoU loss (`IOULoss`, `loss_iou_type`) on positive locations, decoded as `(l, t, r, b)` distances.
- `loss_centerness` — BCE on the FCOS centerness branch.
- `loss_mask` — dice loss between sigmoid mask logits and the strided GT bitmask, computed only over gathered positive instances.
- `loss_sem` *(optional)* — auxiliary focal loss on a coarse semantic segmentation map, only if `mask_branch_sem_loss_on=True`.

---

## Checkpointing & resuming

`train_model()` saves to `output_dir/checkpoints/epoch_XXXX.pt`, containing `model_state_dict`, `optimizer_state_dict`, `epoch`, and `cfg` (saved via `RPDCFG.to_dict()`, i.e. as a plain dict — see [Load a trained model from a checkpoint](#4-load-a-trained-model-from-a-checkpoint) for rebuilding a model from it). Resuming is controlled entirely through the config:

```python
cfg.resume_training = True
cfg.resume_checkpoint_path = "./rpdcnn_output/checkpoints/epoch_0030.pt"  # or "" for auto-latest
```

---

## Output format

Both `predict()` and `infer()` return/save a JSON-serializable dict:

```json
{
  "image_path": "sample.jpg",
  "width": 1920,
  "height": 1080,
  "instances": [
    {
      "class_id": 0,
      "score": 0.87,
      "polygon": [[123.4, 88.1], [140.2, 90.7], "..."]
    }
  ]
}
```

Polygon coordinates in this top-level output are **absolute pixels at the original image resolution**; internally (`out["polygons"]` from a raw `forward()` call) they are normalized `[0, 1]`.

---

## References

RPDCNN's detection/mask wiring is explicitly modeled on:

- Tian et al., *FCOS: Fully Convolutional One-Stage Object Detection*
- Tian et al., *CondInst: Conditional Convolutions for Instance Segmentation*
- [AdelaiDet](https://github.com/aim-uofa/AdelaiDet) — the reference implementation this repo's mask-branch/mask-head/assigner logic mirrors
- Woo et al., *CBAM: Convolutional Block Attention Module*
- Tan et al., *EfficientDet* (BiFPN)

---

*This README was written by inspecting the repository's source (`src/rpdcnn/`) directly, since the upstream repo currently ships only a placeholder `README.md`, no license file, and no repo description.*
