Fix GTA-UAV evaluation and loss (critical: false negatives + wrong R@K)
PROBLEM: GTA-UAV has overlapping satellite crops (partial IoU). Standard InfoNCE with diagonal targets treated valid matches as negatives. R@K checked only diagonal — missed valid matches, artificially low recall. FIXES: 1. WeightedInfoNCE loss (src/losses/weighted_infonce.py): - Per-sample adaptive label smoothing from positive_weights (IoU) - Higher weight → sharper target, lower → softer (semi-positive tolerance) - Based on Game4Loc reference implementation 2. Multi-match R@K evaluation: - Uses dataset.get_all_valid_sat_names() to get ALL valid matches per query - R@K counts hit if ANY valid satellite is in top-K (not just diagonal) - AP computed as MRR over first valid match 3. Dataset returns positive_weight per sample: - Sampled satellite weight passed to loss for adaptive smoothing - All valid satellite candidates exposed for evaluation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -210,6 +210,14 @@ class GTAUAVDataset(Dataset):
|
||||
return transform(rgb)
|
||||
return torch.tensor(0) # placeholder if no transform
|
||||
|
||||
def get_all_valid_sat_names(self) -> list[list[str]]:
|
||||
"""Return all valid satellite matches per drone query (for evaluation).
|
||||
|
||||
In GTA-UAV, each drone has multiple valid satellite crops (partial IoU).
|
||||
Standard diagonal R@K is wrong — must check if ANY valid match is in top-K.
|
||||
"""
|
||||
return [entry["sat_candidates"] for entry in self.entries]
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self.entries)
|
||||
|
||||
@@ -220,13 +228,17 @@ class GTAUAVDataset(Dataset):
|
||||
|
||||
# Sample satellite match (weighted if semi-positive).
|
||||
if entry["sat_weights"] is not None:
|
||||
sat_name = self._rng.choices(
|
||||
entry["sat_candidates"],
|
||||
sat_idx = self._rng.choices(
|
||||
range(len(entry["sat_candidates"])),
|
||||
weights=entry["sat_weights"],
|
||||
k=1,
|
||||
)[0]
|
||||
sat_name = entry["sat_candidates"][sat_idx]
|
||||
pos_weight = entry["sat_weights"][sat_idx]
|
||||
else:
|
||||
sat_name = self._rng.choice(entry["sat_candidates"])
|
||||
sat_idx = self._rng.randrange(len(entry["sat_candidates"]))
|
||||
sat_name = entry["sat_candidates"][sat_idx]
|
||||
pos_weight = 1.0 # strict positive
|
||||
|
||||
sat_img = self._load_image(entry["sat_dir"], sat_name, self.sat_transform)
|
||||
|
||||
@@ -253,6 +265,8 @@ class GTAUAVDataset(Dataset):
|
||||
"sat_caption_l2": sat_l2,
|
||||
"sat_caption_l3": sat_l3,
|
||||
"pair_id": entry["drone_name"],
|
||||
"sat_name": sat_name,
|
||||
"positive_weight": pos_weight,
|
||||
}
|
||||
|
||||
|
||||
@@ -270,4 +284,6 @@ def collate_gtauav_batch(
|
||||
"sat_caption_l2": [b["sat_caption_l2"] for b in batch],
|
||||
"sat_caption_l3": [b["sat_caption_l3"] for b in batch],
|
||||
"pair_ids": [b["pair_id"] for b in batch],
|
||||
"sat_names": [b["sat_name"] for b in batch],
|
||||
"positive_weights": torch.tensor([b["positive_weight"] for b in batch], dtype=torch.float32),
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user