Simplify model: shared DINOv3 WEB + MONA in last 12/24 blocks
Three related architecture changes, driven by a cost/simplicity trade-off: 1. **Shared encoder**: one DINOv3 LVD-1689M (WEB) processes both drone and satellite images. Previously asymmetric — separate WEB (drone) and SAT-493M (satellite) encoders. Saves ~303M frozen params and halves VRAM for the image tower. Expected to lose some satellite-domain inductive bias; MONA adapters pick up the slack. 2. **MONA in last 12/24 blocks**: adapters injected only in the top half of the ViT. The lowest 12 blocks keep their pretrained features untouched. Trainable MONA count drops from 14.0M (48 adapters × 2 encoders) to 3.5M (24 adapters × 1 encoder). 3. **No DINO_SAT**: `nn_models/DINO_SAT` is no longer loaded by the default config. It stays on disk and the path param is kept for backward compat with asymmetric checkpoints. Parameter counts (with text fusion + LoRA + gates): Before: 17.6M trainable / 733M total (2.35%) After: 7.06M trainable / 434M total (1.63%) Also fixes a pre-existing resume bug: checkpoints now record `shared_encoder`, `baseline_mode`, `mona_bottleneck`, `mona_last_n_blocks` so `AsymmetricEncoder.load_checkpoint` can rebuild the right architecture. Old checkpoints still load (missing keys fall back to asymmetric defaults via `ckpt.get(..., <default>)`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -540,6 +540,8 @@ class AsymmetricEncoder(nn.Module):
|
||||
lrsclip_path=lrsclip_path,
|
||||
baseline_mode=ckpt.get("baseline_mode", False),
|
||||
shared_encoder=ckpt.get("shared_encoder", False),
|
||||
mona_bottleneck=ckpt.get("mona_bottleneck", 64),
|
||||
mona_last_n_blocks=ckpt.get("mona_last_n_blocks", 24),
|
||||
device=device,
|
||||
)
|
||||
model.load_state_dict(ckpt["model_state"], strict=False)
|
||||
|
||||
Reference in New Issue
Block a user