a499fcfd65
Six critical fixes to the caption-test training/eval stack:
1. **IndentationError blocker** (train_gtauav.py:765-766)
Unparseable file — train-recall LOGGER.info block was orphaned outside
its `if eval_every` guard. Wrapped in `if train_recall:` so val eval
and Grad-CAM only run on eval epochs.
2. **Full satellite gallery in `_evaluate`**
Old code assembled gallery from DataLoader batches (one random sat per
drone), producing an incomplete gallery of size ≈ N_query instead of
N_unique_sat. Metrics were inflated because retrieval was against a
subset that always contained the target.
New `GTAUAVSatGallery` / `GTAUAVDroneQuery` iterate all unique tiles
and queries independently; full-gallery multi-match R@K + MRR.
3. **Per-sample caption mask** (`AsymmetricEncoder._fuse_with_mask`)
Mixed batches (some samples have captions, some don't) previously
encoded empty strings through DGTRS and mixed the noise output into
every sample via scalar gate. New `encode_query`/`encode_gallery` use
`torch.where` to fall back to pure image features for empty-caption
samples. Training `forward()` routes through the same helper so
training and eval share code.
4. **Symmetric InfoNCE as primary loss** (multi_infonce.InfoNCELoss)
Switched gin default from `WeightedInfoNCELoss` (adaptive label
smoothing — not the Game4Loc soft-IoU target it claimed) to the
existing symmetric InfoNCE with q2g=0.6/g2q=0.4 weighting. Loss type
now selectable via `cfg.loss_type ∈ {"symmetric", "weighted"}`.
5. **MutuallyExclusiveSampler** (new file)
BatchSampler that greedily packs drones whose `sat_candidates` sets
are pairwise disjoint within a batch. Eliminates false negatives from
the semi-positive graph without needing soft-label losses.
At bs=8 keeps 100% of 24,891 train entries; at bs=64 keeps 92.6%.
`set_epoch()` for reproducibility + different batches per epoch.
6. **Temperature clamp [0.01, 0.1]** (both loss modules)
Old tau_max=0.5 allowed the logit distribution to collapse into a
near-uniform softmax. Tightened to the CLIP-standard range.
Also:
- Added `scripts/smoke_eval.py` / `scripts/smoke_train.py` for fast
regression checks (eval in ~2 min, 2 train steps in ~1 min on RTX 4090).
- CLAUDE.md updated to reflect the new pipeline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
59 lines
1.8 KiB
Plaintext
59 lines
1.8 KiB
Plaintext
# GTA-UAV Balanced: Asymmetric DINOv3 (WEB+SAT) with L1/L2/L3 captions.
|
|
# Symmetric InfoNCE + MutuallyExclusiveSampler (no false negatives).
|
|
# 10 epochs, MONA all 24 blocks, 1024-dim retrieval, hard negative bank.
|
|
|
|
import src.losses.multi_infonce
|
|
|
|
# ---- Training ----
|
|
TrainConfigGTAUAV.epochs = 10
|
|
TrainConfigGTAUAV.batch_size = 8
|
|
TrainConfigGTAUAV.num_workers = 4
|
|
TrainConfigGTAUAV.learning_rate = 1e-4
|
|
TrainConfigGTAUAV.text_lr_factor = 0.1
|
|
TrainConfigGTAUAV.weight_decay = 1e-4
|
|
TrainConfigGTAUAV.grad_clip = 1.0
|
|
TrainConfigGTAUAV.grad_accum_steps = 8
|
|
TrainConfigGTAUAV.use_amp = True
|
|
TrainConfigGTAUAV.eval_every = 1
|
|
TrainConfigGTAUAV.warmup_epochs = 2
|
|
TrainConfigGTAUAV.seed = 42
|
|
TrainConfigGTAUAV.device = "cuda"
|
|
|
|
# ---- Model ----
|
|
TrainConfigGTAUAV.init_gate = 0.7
|
|
TrainConfigGTAUAV.baseline_mode = False
|
|
TrainConfigGTAUAV.shared_encoder = False
|
|
TrainConfigGTAUAV.gradient_checkpointing = True
|
|
|
|
# ---- Loss ----
|
|
TrainConfigGTAUAV.loss_type = "symmetric"
|
|
TrainConfigGTAUAV.tau_init = 0.07
|
|
TrainConfigGTAUAV.label_smoothing = 0.1
|
|
TrainConfigGTAUAV.learnable_temperature = True
|
|
TrainConfigGTAUAV.weight_q2g = 0.6
|
|
TrainConfigGTAUAV.weight_g2q = 0.4
|
|
TrainConfigGTAUAV.neg_bank_size = 4096
|
|
|
|
# ---- Sampling ----
|
|
TrainConfigGTAUAV.use_mutex_sampler = True
|
|
|
|
# ---- Output ----
|
|
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text"
|
|
|
|
# ---- Tracking ----
|
|
TrainConfigGTAUAV.use_wandb = False
|
|
TrainConfigGTAUAV.use_tb = True
|
|
TrainConfigGTAUAV.use_gradcam = True
|
|
TrainConfigGTAUAV.gradcam_every = 5
|
|
TrainConfigGTAUAV.use_profiler = False
|
|
TrainConfigGTAUAV.log_grad_norms = True
|
|
|
|
# ---- InfoNCELoss (gin-configurable) ----
|
|
InfoNCELoss.temperature_init = 0.07
|
|
InfoNCELoss.learnable_temperature = True
|
|
InfoNCELoss.label_smoothing = 0.1
|
|
InfoNCELoss.weight_q2g = 0.6
|
|
InfoNCELoss.weight_g2q = 0.4
|
|
InfoNCELoss.tau_min = 0.01
|
|
InfoNCELoss.tau_max = 0.1
|