pikaliov
|
a39f8a9655
|
Add MONA adapters for DINOv3 + LoRA for DGTRS-CLIP text encoder
MONA (Multi-Cognitive One-Shot Nested Adaptation, CVPR 2025):
- 2 adapters per DINOv3 block (after MSA, after MLP) × 24 layers × 2 encoders
- MonaOp: parallel DWConv 3×3/5×5/7×7 + 1×1 projector
- ScaledLayerNorm + down(1024→64) + MonaOp + GELU + up(64→1024)
- 7M params per encoder, 14M total for drone+sat
LoRA (Low-Rank Adaptation):
- Q and V projections in all 12 DGTRS-CLIP transformer blocks
- rank=4, ~147K params total
- Replaces partial unfreeze (was ~7.6M for last block)
Both adapters run in fp32 (torch.amp.autocast disabled) to avoid
AMP gradient overflow. Frozen backbone layers still run in fp16.
Total trainable: 17.6M / 748M (2.35%)
MONA (2×DINOv3): 14.0M
LoRA (DGTRS): 147K
TextFusionMLP: 3.4M
Gates + logit: 3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-21 19:24:01 +03:00 |
|