rpdcnn 0.1.0

RPDCNN: a multi-task perception model for detection and polygon-aware segmentation

RPDCNN

RPDCNN (Region Proposal / Dynamic Convolution Neural Network) is a single-stage, anchor-free, multi-task perception model that performs object detection + instance segmentation on a single forward pass, and emits polygon-shaped masks (YOLO-seg format) rather than raw pixel blobs.

It's built as an FCOS-style detector with a CondInst-style dynamic ("controller-generated") mask head — the wiring intentionally mirrors AdelaiDet's CondInst implementation — with a BiFPN neck and optional CBAM attention bolted on for extra feature quality.

No description, license, or topics are set on the upstream GitHub repo at the time of writing. This README was generated by reading the source directly.

---

Table of contents

• Why RPDCNN
• Architecture
• Repository layout
• Installation
• Quickstart
• Loading a model from a checkpoint
• Label format
• Configuration
• Component reference
• Losses
• Checkpointing & resuming
• Output format
• References

---

Why RPDCNN

Most instance-segmentation stacks (Mask R-CNN, etc.) are two-stage: propose boxes, then crop-and-segment. RPDCNN instead:

1. Regresses boxes and generates a small set of per-instance "dynamic" convolution weights at every FCOS point location, in one shot (no RoIAlign, no proposal stage).
2. Applies those per-instance weights to a shared, high-resolution mask-feature map to produce a dense mask for each detected instance (the CondInst trick).
3. Converts every dense mask to a polygon (cv2.findContours + approxPolyDP) at inference time, so the model's public output is directly usable as YOLO-seg labels.

Training only ever supervises dense bitmasks (dice loss) — the polygon step is a post-processing convenience on the output side, not part of the training loop.

---

Architecture

                                   ┌───────────────────────────┐
                                   │          IMAGE            │
                                   │   (B, 3, img_h, img_w)     │
                                   └─────────────┬─────────────┘
                                                 │
                                                 ▼
                                   ┌───────────────────────────┐
                                   │         BACKBONE          │
                                   │   timm, features_only     │
                                   │ convnext_small / resnet50 │
                                   │ efficientnet_b3 / mnetv3  │
                                   │      / regnety_008        │
                                   └─────────────┬─────────────┘
                                                 │  C_i @ in_strides (e.g. [4,8,16,32])
                                                 ▼
                                   ┌───────────────────────────┐
                                   │           NECK            │
                                   │   BiFPN (learned fusion)  │
                                   │      or torchvision FPN   │
                                   │        (+P6/P7 extras)    │
                                   └─────────────┬─────────────┘
                                                 │  P_i  (named "p{log2(stride)}")
                                                 ▼
                                   ┌───────────────────────────┐
                                   │      CBAM  (optional)     │
                                   │  channel-attn + spatial-  │
                                   │   attn, applied per level │
                                   └─────────────┬─────────────┘
                                                 │
                          ┌──────────────────────┴───────────────────────┐
                          ▼                                              ▼
           ┌───────────────────────────┐                  ┌───────────────────────────┐
           │     DynamicDetHead         │                 │        MaskBranch          │
           │   (applied per level)      │                 │  fuses a subset of levels  │
           │                            │                 │  (e.g. p3,p4,p5[,p6])      │
           │  cls_tower  → cls_logits   │                 │  refine + top-down add     │
           │  bbox_tower → bbox_preds   │                 │  + conv tower              │
           │            → centerness   │                 │      → mask_feats          │
           │            → controller   │                 │  (shared feature map,      │
           │        (dynamic weights)  │                 │   stride = mask_branch_    │
           └─────────────┬─────────────┘                 │        out_stride)         │
                         │                                └─────────────┬─────────────┘
                         │                                              │
             (training)  ▼                                              │
           ┌───────────────────────────┐                                │
           │       FCOSAssigner        │                                │
           │  center-sampling + size-  │                                │
           │  of-interest range match  │                                │
           │   GT box/mask → point     │                                │
           └─────────────┬─────────────┘                                │
                         │ matched_gt_inds (positives)                  │
                         ▼                                              │
           ┌───────────────────────────┐                                │
           │  gather positive points   │                                │
           │  across all levels/images │                                │
           │  → controller params      │◄───────────────────────────────┘
           └─────────────┬─────────────┘
                         │
                         ▼
           ┌───────────────────────────┐
           │      DynamicMaskHead       │
           │  per-instance controller  │
           │  params → conv weights,   │
           │  applied over mask_feats  │
           │  (+ relative coords)      │
           │      → mask_logits        │
           │   (dense, per instance)   │
           └─────────────┬─────────────┘
                         │
        ┌────────────────┴─────────────────┐
        ▼ training                          ▼ inference
 ┌─────────────────┐            ┌─────────────────────────────┐
 │  Dice + BCE loss │            │  sigmoid → threshold        │
 │  vs GT bitmasks  │            │  cv2.findContours +         │
 │  (strided from   │            │  approxPolyDP               │
 │  full-res raster)│            │  → normalized YOLO polygon  │
 └─────────────────┘            └─────────────────────────────┘

Data flow summary: image → Backbone → Neck (BiFPN/FPN) → CBAM → { DynamicDetHead (per level) , MaskBranch (fused levels) } → FCOSAssigner (train) → gather positives → DynamicMaskHead → dense masks → (inference only) polygons.

---

Repository layout

RPDCNN/
├── README.md
├── pyproject.toml
├── requirements.txt
└── src/
    └── rpdcnn/
        ├── __init__.py              # public API: RPDCNN, RPDCFG, get_default_cfg, get_preset_cfg
        ├── cfg.py                   # RPDCFG dataclass + "nano" / "fast" / "main" presets
        ├── rpdcnn.py                # top-level nn.Module: forward(), train_model(), predict(), infer()
        ├── feature_extractor/
        │   ├── backbone.py          # timm wrapper (features_only=True)
        │   ├── bifpn.py             # weighted bidirectional FPN
        │   ├── fpn.py               # torchvision FeaturePyramidNetwork wrapper (+P6/P7)
        │   └── cbam.py              # channel + spatial attention block
        ├── heads/
        │   ├── dynamic_det_head.py  # FCOS-style cls/box/centerness + controller head
        │   ├── mask_branch.py       # fuses neck levels into a shared mask feature map
        │   └── dynamic_mask_head.py # applies per-instance dynamic conv weights
        └── utils/
            ├── fcos_assigner.py     # GT-to-point assignment (center sampling, size-of-interest ranges)
            ├── losses.py            # IOULoss (GIoU/IoU), focal loss, compute_fcos_losses
            ├── mask_utils.py        # aligned_bilinear, dice_coefficient, parse_dynamic_params, etc.
            ├── yolo_poly_dataset.py # YOLO-seg dataset + collate_fn
            └── viz_utils.py         # per-epoch GT-vs-prediction visualization panels

---

Installation

Requires Python ≥ 3.10.

git clone https://github.com/iampa1teja/RPDCNN.git
cd RPDCNN
pip install -e .

or, without an editable install:

pip install -r requirements.txt

Dependencies (from pyproject.toml / requirements.txt):

Package	Version
numpy	>=1.24
opencv-python-headless	>=4.8
timm	>=1.0.0
torch	>=2.1
torchvision	>=0.16

---

Quickstart

1. Build a model from a preset

from rpdcnn import get_preset_cfg, RPDCNN

# presets: "nano" (mobilenetv3, fast/light), "fast" (resnet50), "main" (convnext_small, full accuracy)
cfg = get_preset_cfg("main", num_classes=3)
model = RPDCNN(cfg)

Or start from defaults and override fields manually:

from rpdcnn import get_default_cfg, RPDCNN

cfg = get_default_cfg()
cfg.num_classes = 5
cfg.img_h = cfg.img_w = 640
model = RPDCNN(cfg)

2. Train on a YOLO-segmentation dataset

model.train_model(
    img_dir="data/images",
    label_dir="data/labels",
    format="yolo_poly",              # only supported format right now
    output_dir="./rpdcnn_output",
    num_epochs=50,
    batch_size=4,
    lr=1e-4,
    weight_decay=1e-4,
    num_workers=4,
    save_freq=5,
    class_names=["beaker", "test_tube", "others"],
)

Every epoch this:

• runs one full pass over the dataset and backprops the summed loss dict,
• writes a GT-vs-prediction visualization panel to output_dir/viz/.

Every save_freq epochs (and at the final epoch) it saves a checkpoint to output_dir/checkpoints/epoch_XXXX.pt (model + optimizer + cfg).

To resume, set on the config before constructing/training:

cfg.resume_training = True
cfg.resume_checkpoint_path = ""   # empty -> auto-picks latest checkpoint under output_dir/checkpoints/

3. Run inference on a single image

result, viz = model.predict(
    model,
    image_path="sample.jpg",
    score_thresh=0.3,
    nms_iou_threshold=0.5,
    mask_threshold=0.5,
    approx_epsilon_frac=0.005,
)

print(result["instances"])   # [{"class_id":..., "score":..., "polygon":[[x,y],...]}, ...]

result["instances"][i]["polygon"] is a list of absolute pixel (x, y) coordinates at the image's original resolution.

4. Load a trained model from a checkpoint

Checkpoints saved by train_model() bundle cfg alongside the weights (see Checkpointing & resuming), so a model can be rebuilt without redeclaring RPDCFG by hand. The saved cfg is a plain dict (RPDCFG.to_dict()), not an RPDCFG instance — it must be converted back with RPDCFG.from_dict() before it's passed into RPDCNN(...), since the model reads config fields as attributes (cfg.num_classes, etc.), not dict keys.

import torch
from rpdcnn import RPDCNN
from rpdcnn.cfg import RPDCFG

ckpt_path = "last.pt"
device = "cuda" if torch.cuda.is_available() else "cpu"

ckpt = torch.load(ckpt_path, map_location=device)

cfg = ckpt["cfg"]
if isinstance(cfg, dict):
    cfg = RPDCFG.from_dict(cfg)   # checkpoint stores cfg as a plain dict — convert back

model = RPDCNN(cfg)                              # rebuild architecture from saved cfg
model.load_state_dict(ckpt["model_state_dict"])  # load trained weights
model.to(device)
model.eval()

result, viz = model.predict(image_path="img.jpg")
print(result)
print(viz)

5. Run inference over a folder and save JSON + overlays

model.infer(
    img_dir="data/test_images",
    output_dir="./rpdcnn_infer_output",
    score_thresh=0.3,
    nms_iou_threshold=0.5,
    mask_threshold=0.5,
)

This writes, per image:

rpdcnn_infer_output/
├── json/<stem>.json   # {"image_path","width","height","instances":[...]}
└── viz/<stem>.png     # original image with mask overlay + polygon outline + class legend

6. Raw forward pass (custom decode pipeline)

model.eval()
out = model(images)  # images: (B, 3, img_h, img_w) tensor, already resized

out["polygons"]           # built-in decoded output
out["bbox_preds"]          # (B, N, 4) raw per-point box regressions
out["cls_logits"]          # (B, N, num_classes)
out["centerness_logits"]   # (B, N, 1)
out["controller"]          # (B, N, controller_dim)
out["mask_feats"]          # shared mask feature map
out["points_flat"], out["feature_shapes"]

---

Label format (YOLO-seg)

YoloPolyDataset expects:

img_dir/
├── image_0001.jpg
└── image_0002.png
label_dir/
├── image_0001.txt
└── image_0002.txt

Each .txt has one line per instance:

class_id x1 y1 x2 y2 x3 y3 ... xn yn

All coordinates are normalized to [0, 1]. Each line is one ring / one instance (multi-ring instances aren't supported by the plain YOLO-seg format). Internally, polygons are rasterized once, at full input resolution, into a dense bitmask (_add_bitmasks, mirroring AdelaiDet's CondInst.add_bitmasks); stride-specific targets are obtained by pure strided subsampling of that full-res mask — never a separate resize.

---

Configuration

All knobs live in the RPDCFG dataclass (cfg.py):

Field	Default	Notes
num_classes	80
in_strides	[4, 8, 16, 32]	backbone levels to extract
img_h / img_w	512 / 512
backbone_name	convnext_small	one of convnext_small, resnet50, efficientnet_b3, mobilenetv3_large_100, regnety_008
backbone_pretrained	True
freeze_backbone / freeze_backbone_bn	False
neck_type	bifpn	bifpn or fpn (torchvision, adds P6/P7 automatically)
bifpn_out_channels / bifpn_num_layers	256 / 3
use_cbam / cbam_reduction	True / 16
det_num_convs / det_norm_groups	4 / 8	detection head tower depth
mask_branch_in_features	["p3","p4","p5"]	which neck levels feed the mask branch
mask_branch_channels / out_channels / num_convs	128 / 8 / 4
mask_branch_out_stride	8	stride of the shared mask feature map
mask_head_num_layers / channels	3 / 8	dynamic conv stack
mask_head_out_stride	4	final mask resolution stride
mask_head_sizes_of_interest	[64,128,256]	relative-coord normalization per FPN level
assigner_center_sampling_radius	1.5	FCOS center sampling
assigner_regress_ranges	[(-1,64),(64,128),(128,100000)]	per-level box-size assignment ranges — must have one entry per neck level
loss_focal_alpha / loss_focal_gamma	0.25 / 2.0	classification focal loss
loss_iou_type	giou	box regression loss (iou or giou)

Built-in presets (get_preset_cfg(name, **overrides))

Preset	Backbone	Neck	CBAM	Intended use
nano	mobilenetv3_large_100 (frozen)	fpn	off	fastest / lowest memory
fast	resnet50	bifpn (192ch)	on	balanced speed/accuracy
main	convnext_small	bifpn (256ch, 5 levels)	on	full accuracy

Overrides are validated against RPDCFG's field set and applied on top of the preset (dataclasses.replace), so unset fields fall back to RPDCFG's own defaults rather than the preset silently resetting them.

---

Component reference

• Backbone (feature_extractor/backbone.py) — thin timm.create_model(..., features_only=True) wrapper; picks out only the feature levels matching in_strides and exposes their channel counts.
• BiFPN (feature_extractor/bifpn.py) — learned top-down + bottom-up weighted fusion (BiFPNLayer) over depthwise-separable convs, repeated bifpn_num_layers times.
• FPN (feature_extractor/fpn.py) — alternative neck using torchvision.ops.FeaturePyramidNetwork with LastLevelP6P7 extras.
• CBAM (feature_extractor/cbam.py) — sequential channel-attention (avg+max pool → shared MLP) then spatial-attention (channel avg+max → 7×7 conv), applied per neck level.
• DynamicDetHead (heads/dynamic_det_head.py) — FCOS-style head: separate cls/bbox towers, per-level learnable Scale, outputs (bbox_preds, cls_logits, centerness_logits, controller).
• MaskBranch (heads/mask_branch.py) — refines and top-down-fuses the chosen neck levels into one shared mask_feats tensor; optional auxiliary semantic-segmentation loss (mask_branch_sem_loss_on).
• DynamicMaskHead (heads/dynamic_mask_head.py) — CondInst-style controller-conv head: unpacks each instance's controller vector into conv weights/biases (parse_dynamic_params), runs them as grouped 1×1 convs over mask_feats + relative-coordinate channels, upsamples via aligned_bilinear.
• FCOSAssigner (utils/fcos_assigner.py) — assigns GT boxes to point locations per level using center sampling + per-level regress ranges.

---

Losses

Returned as a dict from forward() in training mode and summed for backprop:

• loss_cls — sigmoid focal loss on classification logits.
• loss_bbox — IoU/GIoU loss (IOULoss, loss_iou_type) on positive locations, decoded as (l, t, r, b) distances.
• loss_centerness — BCE on the FCOS centerness branch.
• loss_mask — dice loss between sigmoid mask logits and the strided GT bitmask, computed only over gathered positive instances.
• loss_sem (optional) — auxiliary focal loss on a coarse semantic segmentation map, only if mask_branch_sem_loss_on=True.

---

Checkpointing & resuming

train_model() saves to output_dir/checkpoints/epoch_XXXX.pt, containing model_state_dict, optimizer_state_dict, epoch, and cfg (saved via RPDCFG.to_dict(), i.e. as a plain dict — see Load a trained model from a checkpoint for rebuilding a model from it). Resuming is controlled entirely through the config:

cfg.resume_training = True
cfg.resume_checkpoint_path = "./rpdcnn_output/checkpoints/epoch_0030.pt"  # or "" for auto-latest

---

Output format

Both predict() and infer() return/save a JSON-serializable dict:

{
  "image_path": "sample.jpg",
  "width": 1920,
  "height": 1080,
  "instances": [
    {
      "class_id": 0,
      "score": 0.87,
      "polygon": [[123.4, 88.1], [140.2, 90.7], "..."]
    }
  ]
}

Polygon coordinates in this top-level output are absolute pixels at the original image resolution; internally (out["polygons"] from a raw forward() call) they are normalized [0, 1].

---

References

RPDCNN's detection/mask wiring is explicitly modeled on:

• Tian et al., FCOS: Fully Convolutional One-Stage Object Detection
• Tian et al., CondInst: Conditional Convolutions for Instance Segmentation
• AdelaiDet — the reference implementation this repo's mask-branch/mask-head/assigner logic mirrors
• Woo et al., CBAM: Convolutional Block Attention Module
• Tan et al., EfficientDet (BiFPN)

---

This README was written by inspecting the repository's source (src/rpdcnn/) directly, since the upstream repo currently ships only a placeholder README.md, no license file, and no repo description.