PROBLEM: GTA-UAV has overlapping satellite crops (partial IoU).
Standard InfoNCE with diagonal targets treated valid matches as negatives.
R@K checked only diagonal — missed valid matches, artificially low recall.
FIXES:
1. WeightedInfoNCE loss (src/losses/weighted_infonce.py):
- Per-sample adaptive label smoothing from positive_weights (IoU)
- Higher weight → sharper target, lower → softer (semi-positive tolerance)
- Based on Game4Loc reference implementation
2. Multi-match R@K evaluation:
- Uses dataset.get_all_valid_sat_names() to get ALL valid matches per query
- R@K counts hit if ANY valid satellite is in top-K (not just diagonal)
- AP computed as MRR over first valid match
3. Dataset returns positive_weight per sample:
- Sampled satellite weight passed to loss for adaptive smoothing
- All valid satellite candidates exposed for evaluation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Architecture changes:
- Asymmetric DINOv3: WEB (drone) + SAT (satellite) with separate MONA
- MONA on all 24 blocks per encoder (was last 12)
- Remove projection, native 1024-dim retrieval space (was 512)
- Total: 748M params, 17.6M trainable (2.35%)
Hard negative memory bank:
- MoCo-style FIFO queue of 4096 detached gallery embeddings
- Each batch: B in-batch + Q queue negatives in InfoNCE
- Queue updated after each forward pass
Training config:
- batch_size=8, grad_accum=8, effective_batch=64
- eval_every=1 (eval + train recall every epoch)
- Max bs=24 with grad checkpointing on RTX 4090
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Architecture changes:
- Removed proj_drone/proj_sat (1024→512): retrieval space is now
DINOv3 native 1024-dim, no information loss from projection
- TextFusionMLP: 2304→1024→1024 (was 2304→768→512), shared between branches
- Gallery branch now uses satellite captions (L1/L2/L3) via shared TextFusionMLP
- Two separate GatedFusion gates: α_q (query) and α_g (gallery)
- For sat images without captions (~57%): gate passes image features through
Dataset changes:
- GTAUAVDataset now loads satellite captions from caption index
- collate_gtauav_batch includes sat_caption_l1/l2/l3
Training loop:
- Passes satellite captions to model forward
- Logs both gate_q and gate_g values
11.1M trainable / 734M total (1.51%)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: GradScaler scales gradients by ~65536 in fp16, causing
logit_scale.exp() gradient to overflow. The learnable temperature
and similarity logits must stay in fp32.
Fix: model forward runs inside autocast(fp16), but loss computation
(similarity @ temperature + cross_entropy) runs outside in fp32.
Also: clamp logit_scale in logit-space before exp() and force
similarity computation to fp32.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
