Add StripNet backbone option with Conv-MONA adaptation
StripNet-small (Strip-R-CNN, HVision-NKU) as alternative image encoder to
DINOv3 ViT-L/16. ~28M params (10x smaller). Output 512-dim from stage 4
projected to 1024 to keep retrieval space compatible with DINOv3 configs.
- src/models/stripnet/: self-contained backbone (model.py, conv_mona.py).
State-dict naming follows upstream Strip-R-CNN repo (conv_spatial1/2);
ImageNet-1K pretrained head dropped on load.
- Conv-MONA: 2D adaptation of MONA (CVPR 2025) for CNN blocks. BN → 1x1
Down(C->bn) → multi-scale DWConv {3,5,7} mean → +residual → GELU →
1x1 Up(bn->C) with channel-wise layer scale γ init 1e-6. Two adapters
per StripNet Block (post-attn, post-mlp); injected into deepest N stages.
- StripNetEncoder: GAP + Linear(512->1024). Overrides train() to keep
frozen BatchNorm stats stable across mode flips.
- AsymmetricEncoder: new `backbone="stripnet"` option (always shared).
- TrainConfigGTAUAV: backbone, stripnet_path, stripnet_mona_last_n_stages.
- conf/gtauav_balanced_stripnet.gin + gtauav_baseline_stripnet.gin.
Smoke test: forward [2,3,256,256] -> [2,1024]. Trainable: 1.2M baseline
(8.27%), 4.76M with text (3.35% of 142M).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -90,6 +90,10 @@ class TrainConfigGTAUAV:
|
||||
mona_bottleneck: int = 64
|
||||
mona_last_n_blocks: int = 12 # inject adapters only in last 12 of 24 ViT blocks
|
||||
gradient_checkpointing: bool = True # trade compute for VRAM (allows larger batch)
|
||||
# StripNet backbone option (replaces DINOv3 when backbone="stripnet").
|
||||
backbone: str = "dinov3" # "dinov3" or "stripnet"
|
||||
stripnet_path: str = "nn_models/STRIPNET/stripnet_s.pth"
|
||||
stripnet_mona_last_n_stages: int = 2 # Conv-MONA in last N of 4 StripNet stages
|
||||
|
||||
# Training.
|
||||
resume_from: str | None = None # path to checkpoint for resuming
|
||||
@@ -564,7 +568,10 @@ def train(cfg: TrainConfigGTAUAV) -> None:
|
||||
start_epoch = resume_ckpt.get("epoch", -1) + 1
|
||||
else:
|
||||
mode_str = "baseline (no text)" if cfg.baseline_mode else "with text (L1/L2/L3)"
|
||||
enc_str = "shared DINOv3 WEB" if cfg.shared_encoder else "asymmetric (WEB + SAT)"
|
||||
if cfg.backbone == "stripnet":
|
||||
enc_str = "StripNet-small (shared, 512→1024 proj)"
|
||||
else:
|
||||
enc_str = "shared DINOv3 WEB" if cfg.shared_encoder else "asymmetric (WEB + SAT)"
|
||||
LOGGER.info("Building model — %s, %s", mode_str, enc_str)
|
||||
model = AsymmetricEncoder(
|
||||
dino_web_path=cfg.dino_web_path,
|
||||
@@ -576,11 +583,15 @@ def train(cfg: TrainConfigGTAUAV) -> None:
|
||||
mona_bottleneck=cfg.mona_bottleneck,
|
||||
mona_last_n_blocks=cfg.mona_last_n_blocks,
|
||||
device=cfg.device,
|
||||
backbone=cfg.backbone,
|
||||
stripnet_path=cfg.stripnet_path,
|
||||
stripnet_mona_last_n_stages=cfg.stripnet_mona_last_n_stages,
|
||||
).to(cfg.device)
|
||||
LOGGER.info("embed_dim=%d", model.embed_dim)
|
||||
|
||||
# --- Gradient checkpointing (trade compute for VRAM) ---
|
||||
if cfg.gradient_checkpointing:
|
||||
# StripNet doesn't expose set_gradient_checkpointing — skip silently.
|
||||
if cfg.gradient_checkpointing and cfg.backbone == "dinov3":
|
||||
if cfg.shared_encoder:
|
||||
model.image_encoder.set_gradient_checkpointing(True)
|
||||
else:
|
||||
@@ -589,6 +600,10 @@ def train(cfg: TrainConfigGTAUAV) -> None:
|
||||
if model.text_encoder is not None:
|
||||
model.text_encoder.transformer.gradient_checkpointing = True
|
||||
LOGGER.info("Gradient checkpointing enabled (DINOv3 + DGTRS)")
|
||||
elif cfg.gradient_checkpointing and cfg.backbone == "stripnet":
|
||||
if model.text_encoder is not None:
|
||||
model.text_encoder.transformer.gradient_checkpointing = True
|
||||
LOGGER.info("Gradient checkpointing enabled (DGTRS only; StripNet doesn't support)")
|
||||
|
||||
n_trainable = sum(p.numel() for p in model.trainable_parameters())
|
||||
n_total = sum(p.numel() for p in model.parameters())
|
||||
|
||||
Reference in New Issue
Block a user