Fix GTA-UAV evaluation and loss (critical: false negatives + wrong R@K)

PROBLEM: GTA-UAV has overlapping satellite crops (partial IoU).
Standard InfoNCE with diagonal targets treated valid matches as negatives.
R@K checked only diagonal — missed valid matches, artificially low recall.

FIXES:
1. WeightedInfoNCE loss (src/losses/weighted_infonce.py):
   - Per-sample adaptive label smoothing from positive_weights (IoU)
   - Higher weight → sharper target, lower → softer (semi-positive tolerance)
   - Based on Game4Loc reference implementation

2. Multi-match R@K evaluation:
   - Uses dataset.get_all_valid_sat_names() to get ALL valid matches per query
   - R@K counts hit if ANY valid satellite is in top-K (not just diagonal)
   - AP computed as MRR over first valid match

3. Dataset returns positive_weight per sample:
   - Sampled satellite weight passed to loss for adaptive smoothing
   - All valid satellite candidates exposed for evaluation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-24 12:40:10 +03:00
parent 04d5307221
commit b6dccbba7b
4 changed files with 242 additions and 50 deletions

View File

@@ -210,6 +210,14 @@ class GTAUAVDataset(Dataset):
return transform(rgb)
return torch.tensor(0) # placeholder if no transform
def get_all_valid_sat_names(self) -> list[list[str]]:
"""Return all valid satellite matches per drone query (for evaluation).
In GTA-UAV, each drone has multiple valid satellite crops (partial IoU).
Standard diagonal R@K is wrong — must check if ANY valid match is in top-K.
"""
return [entry["sat_candidates"] for entry in self.entries]
def __len__(self) -> int:
return len(self.entries)
@@ -220,13 +228,17 @@ class GTAUAVDataset(Dataset):
# Sample satellite match (weighted if semi-positive).
if entry["sat_weights"] is not None:
sat_name = self._rng.choices(
entry["sat_candidates"],
sat_idx = self._rng.choices(
range(len(entry["sat_candidates"])),
weights=entry["sat_weights"],
k=1,
)[0]
sat_name = entry["sat_candidates"][sat_idx]
pos_weight = entry["sat_weights"][sat_idx]
else:
sat_name = self._rng.choice(entry["sat_candidates"])
sat_idx = self._rng.randrange(len(entry["sat_candidates"]))
sat_name = entry["sat_candidates"][sat_idx]
pos_weight = 1.0 # strict positive
sat_img = self._load_image(entry["sat_dir"], sat_name, self.sat_transform)
@@ -253,6 +265,8 @@ class GTAUAVDataset(Dataset):
"sat_caption_l2": sat_l2,
"sat_caption_l3": sat_l3,
"pair_id": entry["drone_name"],
"sat_name": sat_name,
"positive_weight": pos_weight,
}
@@ -270,4 +284,6 @@ def collate_gtauav_batch(
"sat_caption_l2": [b["sat_caption_l2"] for b in batch],
"sat_caption_l3": [b["sat_caption_l3"] for b in batch],
"pair_ids": [b["pair_id"] for b in batch],
"sat_names": [b["sat_name"] for b in batch],
"positive_weights": torch.tensor([b["positive_weight"] for b in batch], dtype=torch.float32),
}