Fix GTA-UAV eval + training pipeline: full gallery, mutex sampler, per-sample mask
Six critical fixes to the caption-test training/eval stack:
1. **IndentationError blocker** (train_gtauav.py:765-766)
Unparseable file — train-recall LOGGER.info block was orphaned outside
its `if eval_every` guard. Wrapped in `if train_recall:` so val eval
and Grad-CAM only run on eval epochs.
2. **Full satellite gallery in `_evaluate`**
Old code assembled gallery from DataLoader batches (one random sat per
drone), producing an incomplete gallery of size ≈ N_query instead of
N_unique_sat. Metrics were inflated because retrieval was against a
subset that always contained the target.
New `GTAUAVSatGallery` / `GTAUAVDroneQuery` iterate all unique tiles
and queries independently; full-gallery multi-match R@K + MRR.
3. **Per-sample caption mask** (`AsymmetricEncoder._fuse_with_mask`)
Mixed batches (some samples have captions, some don't) previously
encoded empty strings through DGTRS and mixed the noise output into
every sample via scalar gate. New `encode_query`/`encode_gallery` use
`torch.where` to fall back to pure image features for empty-caption
samples. Training `forward()` routes through the same helper so
training and eval share code.
4. **Symmetric InfoNCE as primary loss** (multi_infonce.InfoNCELoss)
Switched gin default from `WeightedInfoNCELoss` (adaptive label
smoothing — not the Game4Loc soft-IoU target it claimed) to the
existing symmetric InfoNCE with q2g=0.6/g2q=0.4 weighting. Loss type
now selectable via `cfg.loss_type ∈ {"symmetric", "weighted"}`.
5. **MutuallyExclusiveSampler** (new file)
BatchSampler that greedily packs drones whose `sat_candidates` sets
are pairwise disjoint within a batch. Eliminates false negatives from
the semi-positive graph without needing soft-label losses.
At bs=8 keeps 100% of 24,891 train entries; at bs=64 keeps 92.6%.
`set_epoch()` for reproducibility + different batches per epoch.
6. **Temperature clamp [0.01, 0.1]** (both loss modules)
Old tau_max=0.5 allowed the logit distribution to collapse into a
near-uniform softmax. Tightened to the CLIP-standard range.
Also:
- Added `scripts/smoke_eval.py` / `scripts/smoke_train.py` for fast
regression checks (eval in ~2 min, 2 train steps in ~1 min on RTX 4090).
- CLAUDE.md updated to reflect the new pipeline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,8 @@
|
||||
# GTA-UAV Balanced: Asymmetric DINOv3 (WEB+SAT) with L1/L2/L3 captions.
|
||||
# WeightedInfoNCE loss for GTA-UAV partial overlap handling.
|
||||
# Symmetric InfoNCE + MutuallyExclusiveSampler (no false negatives).
|
||||
# 10 epochs, MONA all 24 blocks, 1024-dim retrieval, hard negative bank.
|
||||
|
||||
import src.losses.weighted_infonce
|
||||
import src.losses.multi_infonce
|
||||
|
||||
# ---- Training ----
|
||||
TrainConfigGTAUAV.epochs = 10
|
||||
@@ -26,11 +26,17 @@ TrainConfigGTAUAV.shared_encoder = False
|
||||
TrainConfigGTAUAV.gradient_checkpointing = True
|
||||
|
||||
# ---- Loss ----
|
||||
TrainConfigGTAUAV.loss_type = "symmetric"
|
||||
TrainConfigGTAUAV.tau_init = 0.07
|
||||
TrainConfigGTAUAV.label_smoothing = 0.1
|
||||
TrainConfigGTAUAV.learnable_temperature = True
|
||||
TrainConfigGTAUAV.weight_q2g = 0.6
|
||||
TrainConfigGTAUAV.weight_g2q = 0.4
|
||||
TrainConfigGTAUAV.neg_bank_size = 4096
|
||||
|
||||
# ---- Sampling ----
|
||||
TrainConfigGTAUAV.use_mutex_sampler = True
|
||||
|
||||
# ---- Output ----
|
||||
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text"
|
||||
|
||||
@@ -42,8 +48,11 @@ TrainConfigGTAUAV.gradcam_every = 5
|
||||
TrainConfigGTAUAV.use_profiler = False
|
||||
TrainConfigGTAUAV.log_grad_norms = True
|
||||
|
||||
# ---- WeightedInfoNCE (gin-configurable) ----
|
||||
WeightedInfoNCELoss.temperature_init = 0.07
|
||||
WeightedInfoNCELoss.learnable_temperature = True
|
||||
WeightedInfoNCELoss.label_smoothing = 0.1
|
||||
WeightedInfoNCELoss.k = 5.0
|
||||
# ---- InfoNCELoss (gin-configurable) ----
|
||||
InfoNCELoss.temperature_init = 0.07
|
||||
InfoNCELoss.learnable_temperature = True
|
||||
InfoNCELoss.label_smoothing = 0.1
|
||||
InfoNCELoss.weight_q2g = 0.6
|
||||
InfoNCELoss.weight_g2q = 0.4
|
||||
InfoNCELoss.tau_min = 0.01
|
||||
InfoNCELoss.tau_max = 0.1
|
||||
|
||||
Reference in New Issue
Block a user