Add StripNet backbone option with Conv-MONA adaptation

StripNet-small (Strip-R-CNN, HVision-NKU) as alternative image encoder to
DINOv3 ViT-L/16. ~28M params (10x smaller). Output 512-dim from stage 4
projected to 1024 to keep retrieval space compatible with DINOv3 configs.

- src/models/stripnet/: self-contained backbone (model.py, conv_mona.py).
  State-dict naming follows upstream Strip-R-CNN repo (conv_spatial1/2);
  ImageNet-1K pretrained head dropped on load.
- Conv-MONA: 2D adaptation of MONA (CVPR 2025) for CNN blocks. BN → 1x1
  Down(C->bn) → multi-scale DWConv {3,5,7} mean → +residual → GELU →
  1x1 Up(bn->C) with channel-wise layer scale γ init 1e-6. Two adapters
  per StripNet Block (post-attn, post-mlp); injected into deepest N stages.
- StripNetEncoder: GAP + Linear(512->1024). Overrides train() to keep
  frozen BatchNorm stats stable across mode flips.
- AsymmetricEncoder: new `backbone="stripnet"` option (always shared).
- TrainConfigGTAUAV: backbone, stripnet_path, stripnet_mona_last_n_stages.
- conf/gtauav_balanced_stripnet.gin + gtauav_baseline_stripnet.gin.

Smoke test: forward [2,3,256,256] -> [2,1024]. Trainable: 1.2M baseline
(8.27%), 4.76M with text (3.35% of 142M).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-25 14:34:53 +03:00
parent 814586ce3b
commit d4cb2dd300
8 changed files with 487 additions and 4 deletions

View File

@@ -90,6 +90,10 @@ class TrainConfigGTAUAV:
mona_bottleneck: int = 64
mona_last_n_blocks: int = 12 # inject adapters only in last 12 of 24 ViT blocks
gradient_checkpointing: bool = True # trade compute for VRAM (allows larger batch)
# StripNet backbone option (replaces DINOv3 when backbone="stripnet").
backbone: str = "dinov3" # "dinov3" or "stripnet"
stripnet_path: str = "nn_models/STRIPNET/stripnet_s.pth"
stripnet_mona_last_n_stages: int = 2 # Conv-MONA in last N of 4 StripNet stages
# Training.
resume_from: str | None = None # path to checkpoint for resuming
@@ -564,7 +568,10 @@ def train(cfg: TrainConfigGTAUAV) -> None:
start_epoch = resume_ckpt.get("epoch", -1) + 1
else:
mode_str = "baseline (no text)" if cfg.baseline_mode else "with text (L1/L2/L3)"
enc_str = "shared DINOv3 WEB" if cfg.shared_encoder else "asymmetric (WEB + SAT)"
if cfg.backbone == "stripnet":
enc_str = "StripNet-small (shared, 512→1024 proj)"
else:
enc_str = "shared DINOv3 WEB" if cfg.shared_encoder else "asymmetric (WEB + SAT)"
LOGGER.info("Building model — %s, %s", mode_str, enc_str)
model = AsymmetricEncoder(
dino_web_path=cfg.dino_web_path,
@@ -576,11 +583,15 @@ def train(cfg: TrainConfigGTAUAV) -> None:
mona_bottleneck=cfg.mona_bottleneck,
mona_last_n_blocks=cfg.mona_last_n_blocks,
device=cfg.device,
backbone=cfg.backbone,
stripnet_path=cfg.stripnet_path,
stripnet_mona_last_n_stages=cfg.stripnet_mona_last_n_stages,
).to(cfg.device)
LOGGER.info("embed_dim=%d", model.embed_dim)
# --- Gradient checkpointing (trade compute for VRAM) ---
if cfg.gradient_checkpointing:
# StripNet doesn't expose set_gradient_checkpointing — skip silently.
if cfg.gradient_checkpointing and cfg.backbone == "dinov3":
if cfg.shared_encoder:
model.image_encoder.set_gradient_checkpointing(True)
else:
@@ -589,6 +600,10 @@ def train(cfg: TrainConfigGTAUAV) -> None:
if model.text_encoder is not None:
model.text_encoder.transformer.gradient_checkpointing = True
LOGGER.info("Gradient checkpointing enabled (DINOv3 + DGTRS)")
elif cfg.gradient_checkpointing and cfg.backbone == "stripnet":
if model.text_encoder is not None:
model.text_encoder.transformer.gradient_checkpointing = True
LOGGER.info("Gradient checkpointing enabled (DGTRS only; StripNet doesn't support)")
n_trainable = sum(p.numel() for p in model.trainable_parameters())
n_total = sum(p.numel() for p in model.parameters())