Pikaliov/caption-test
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix GTA-UAV evaluation and loss (critical: false negatives + wrong R@K)

Browse Source 
PROBLEM: GTA-UAV has overlapping satellite crops (partial IoU).
Standard InfoNCE with diagonal targets treated valid matches as negatives.
R@K checked only diagonal — missed valid matches, artificially low recall.

FIXES:
1. WeightedInfoNCE loss (src/losses/weighted_infonce.py):
   - Per-sample adaptive label smoothing from positive_weights (IoU)
   - Higher weight → sharper target, lower → softer (semi-positive tolerance)
   - Based on Game4Loc reference implementation

2. Multi-match R@K evaluation:
   - Uses dataset.get_all_valid_sat_names() to get ALL valid matches per query
   - R@K counts hit if ANY valid satellite is in top-K (not just diagonal)
   - AP computed as MRR over first valid match

3. Dataset returns positive_weight per sample:
   - Sampled satellite weight passed to loss for adaptive smoothing
   - All valid satellite candidates exposed for evaluation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-24 12:40:10 +03:00
parent 04d5307221
commit b6dccbba7b
4 changed files with 242 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
conf/gtauav_balanced.gin
View File

@@ -1,11 +1,8 @@
# GTA-UAV Balanced: Asymmetric DINOv3 (WEB+SAT) with L1/L2/L3 captions.
# query = sigma(alpha) * drone + (1-sigma(alpha)) * text -> InfoNCE vs gallery
# WeightedInfoNCE loss for GTA-UAV partial overlap handling.
# 10 epochs, MONA all 24 blocks, 1024-dim retrieval, hard negative bank.
#
# NOTE: TrainConfigGTAUAV is registered by train_gtauav.py before gin parsing.
# InfoNCELoss is registered via import below.
import src.losses.multi_infonce
import src.losses.weighted_infonce
# ---- Training ----
TrainConfigGTAUAV.epochs = 10
@@ -31,8 +28,6 @@ TrainConfigGTAUAV.gradient_checkpointing = True
# ---- Loss ----
TrainConfigGTAUAV.tau_init = 0.07
TrainConfigGTAUAV.label_smoothing = 0.1
TrainConfigGTAUAV.weight_q2g = 0.6
TrainConfigGTAUAV.weight_g2q = 0.4
TrainConfigGTAUAV.learnable_temperature = True
TrainConfigGTAUAV.neg_bank_size = 4096
@@ -47,10 +42,8 @@ TrainConfigGTAUAV.gradcam_every = 5
TrainConfigGTAUAV.use_profiler = False
TrainConfigGTAUAV.log_grad_norms = True
# ---- InfoNCE Loss (gin-configurable) ----
InfoNCELoss.temperature_init = 0.07
InfoNCELoss.temperature_final = 0.01
InfoNCELoss.label_smoothing = 0.1
InfoNCELoss.weight_q2g = 0.6
InfoNCELoss.weight_g2q = 0.4
InfoNCELoss.learnable_temperature = True
# ---- WeightedInfoNCE (gin-configurable) ----
WeightedInfoNCELoss.temperature_init = 0.07
WeightedInfoNCELoss.learnable_temperature = True
WeightedInfoNCELoss.label_smoothing = 0.1
WeightedInfoNCELoss.k = 5.0