d4cb2dd300
StripNet-small (Strip-R-CNN, HVision-NKU) as alternative image encoder to
DINOv3 ViT-L/16. ~28M params (10x smaller). Output 512-dim from stage 4
projected to 1024 to keep retrieval space compatible with DINOv3 configs.
- src/models/stripnet/: self-contained backbone (model.py, conv_mona.py).
State-dict naming follows upstream Strip-R-CNN repo (conv_spatial1/2);
ImageNet-1K pretrained head dropped on load.
- Conv-MONA: 2D adaptation of MONA (CVPR 2025) for CNN blocks. BN → 1x1
Down(C->bn) → multi-scale DWConv {3,5,7} mean → +residual → GELU →
1x1 Up(bn->C) with channel-wise layer scale γ init 1e-6. Two adapters
per StripNet Block (post-attn, post-mlp); injected into deepest N stages.
- StripNetEncoder: GAP + Linear(512->1024). Overrides train() to keep
frozen BatchNorm stats stable across mode flips.
- AsymmetricEncoder: new `backbone="stripnet"` option (always shared).
- TrainConfigGTAUAV: backbone, stripnet_path, stripnet_mona_last_n_stages.
- conf/gtauav_balanced_stripnet.gin + gtauav_baseline_stripnet.gin.
Smoke test: forward [2,3,256,256] -> [2,1024]. Trainable: 1.2M baseline
(8.27%), 4.76M with text (3.35% of 142M).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
28 lines
1.1 KiB
Plaintext
28 lines
1.1 KiB
Plaintext
# GTA-UAV Balanced (StripNet backbone): StripNet-small + Conv-MONA in last 2 stages.
|
||
# Replaces DINOv3 ViT-L/16 with strip-shaped DWConv hierarchical CNN (~28M params,
|
||
# 10× smaller than DINOv3). Output 512-dim → projected to 1024 to match retrieval space.
|
||
#
|
||
# Trainable:
|
||
# - Projection (Linear 512→1024): ~525K
|
||
# - Conv-MONA in stages 3 & 4 (2 adapters per Block × 6 blocks total): ~2-3M
|
||
# - LoRA on DGTRS-CLIP: 147K
|
||
# - TextFusionMLP (shared): ~3.4M
|
||
# - GatedFusion gates + tau: 3 scalars
|
||
#
|
||
# StripNet pretrained on ImageNet-1K (head dropped); state-dict naming follows
|
||
# upstream Strip-R-CNN repo (`conv_spatial1/2`).
|
||
|
||
include 'conf/gtauav_balanced.gin'
|
||
|
||
# ---- Backbone ----
|
||
TrainConfigGTAUAV.backbone = "stripnet"
|
||
TrainConfigGTAUAV.stripnet_path = "nn_models/STRIPNET/stripnet_s.pth"
|
||
TrainConfigGTAUAV.stripnet_mona_last_n_stages = 2 # Conv-MONA in stages 3 & 4 (deepest)
|
||
|
||
# ---- Model overrides ----
|
||
TrainConfigGTAUAV.shared_encoder = True # StripNet always shared (one CNN for both branches)
|
||
TrainConfigGTAUAV.mona_bottleneck = 64 # Conv-MONA bottleneck channels
|
||
|
||
# ---- Output ----
|
||
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text_stripnet"
|