a499fcfd65
Six critical fixes to the caption-test training/eval stack:
1. **IndentationError blocker** (train_gtauav.py:765-766)
Unparseable file — train-recall LOGGER.info block was orphaned outside
its `if eval_every` guard. Wrapped in `if train_recall:` so val eval
and Grad-CAM only run on eval epochs.
2. **Full satellite gallery in `_evaluate`**
Old code assembled gallery from DataLoader batches (one random sat per
drone), producing an incomplete gallery of size ≈ N_query instead of
N_unique_sat. Metrics were inflated because retrieval was against a
subset that always contained the target.
New `GTAUAVSatGallery` / `GTAUAVDroneQuery` iterate all unique tiles
and queries independently; full-gallery multi-match R@K + MRR.
3. **Per-sample caption mask** (`AsymmetricEncoder._fuse_with_mask`)
Mixed batches (some samples have captions, some don't) previously
encoded empty strings through DGTRS and mixed the noise output into
every sample via scalar gate. New `encode_query`/`encode_gallery` use
`torch.where` to fall back to pure image features for empty-caption
samples. Training `forward()` routes through the same helper so
training and eval share code.
4. **Symmetric InfoNCE as primary loss** (multi_infonce.InfoNCELoss)
Switched gin default from `WeightedInfoNCELoss` (adaptive label
smoothing — not the Game4Loc soft-IoU target it claimed) to the
existing symmetric InfoNCE with q2g=0.6/g2q=0.4 weighting. Loss type
now selectable via `cfg.loss_type ∈ {"symmetric", "weighted"}`.
5. **MutuallyExclusiveSampler** (new file)
BatchSampler that greedily packs drones whose `sat_candidates` sets
are pairwise disjoint within a batch. Eliminates false negatives from
the semi-positive graph without needing soft-label losses.
At bs=8 keeps 100% of 24,891 train entries; at bs=64 keeps 92.6%.
`set_epoch()` for reproducibility + different batches per epoch.
6. **Temperature clamp [0.01, 0.1]** (both loss modules)
Old tau_max=0.5 allowed the logit distribution to collapse into a
near-uniform softmax. Tightened to the CLIP-standard range.
Also:
- Added `scripts/smoke_eval.py` / `scripts/smoke_train.py` for fast
regression checks (eval in ~2 min, 2 train steps in ~1 min on RTX 4090).
- CLAUDE.md updated to reflect the new pipeline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
47 lines
1.3 KiB
Python
47 lines
1.3 KiB
Python
"""Smoke-test for the rewritten `_evaluate` function.
|
|
|
|
Loads checkpoint ckpt_epoch005.pt and runs the new full-gallery eval with
|
|
max_batches=5 to verify end-to-end without waiting for a full epoch.
|
|
"""
|
|
from torch.utils.data import DataLoader
|
|
|
|
from src.datasets.gtauav_dataset import GTAUAVDataset, collate_gtauav_batch
|
|
from src.models.asymmetric_encoder import AsymmetricEncoder, get_dino_transform
|
|
from src.training.train_gtauav import _evaluate
|
|
|
|
CKPT = "out/gtauav/with_text/ckpt_epoch005.pt"
|
|
|
|
def main() -> None:
|
|
model, _ = AsymmetricEncoder.load_checkpoint(CKPT, device="cuda")
|
|
|
|
eval_tf = get_dino_transform(image_size=256)
|
|
ds = GTAUAVDataset(
|
|
pair_json="meta/test_20.json",
|
|
filter_meta="meta/seg_filter.json",
|
|
image_transform=eval_tf,
|
|
)
|
|
loader = DataLoader(
|
|
ds,
|
|
batch_size=32,
|
|
shuffle=False,
|
|
num_workers=2,
|
|
collate_fn=collate_gtauav_batch,
|
|
pin_memory=True,
|
|
)
|
|
|
|
print("Running _evaluate (max_batches=5 on queries, full gallery)...")
|
|
metrics = _evaluate(
|
|
model=model,
|
|
loader=loader,
|
|
device="cuda",
|
|
max_batches=5,
|
|
desc="smoke",
|
|
)
|
|
print("--- metrics ---")
|
|
for k, v in metrics.items():
|
|
print(f" {k}: {v:.4f}" if isinstance(v, float) else f" {k}: {v}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|