f8e0631210
Batches assembled from visually-similar drone queries pressure the model to
learn finer discriminative features. Random mutex batches average ~0.26
pairwise cosine similarity in query embedding space; DSS batches average
~0.71 — confirming the lookalikes grouping works as intended.
Algorithm per batch:
1. Pick a random seed drone from the remaining pool.
2. Rank the entire remaining pool by cosine similarity to the seed.
3. Walk the ranking in descending order; add items whose sat_candidates
don't collide with the batch's already-claimed set.
4. Drop the seed if no valid batch can be assembled (rare mutex deadlock).
Inherits MutuallyExclusiveSampler semantics — no false negatives. Degrades
gracefully to mutex-only when no embeddings are set (warmup epochs, or if
`sampler_type="mutex"` is chosen).
Integration in `train_gtauav.py`:
- New `_embed_drone_queries` helper: model.encode_query forwarded over
GTAUAVDroneQuery, returns [N, D] CPU tensor. ~13s per 1024 queries on
a 4090 → ~5 min for the full 25K train set.
- Epoch loop re-embeds every `dss_reembed_every` epochs after a `dss_warmup_epochs`
warmup (first epochs use mutex-only since untrained embeddings aren't
informative for kNN).
- Config: `sampler_type` ∈ {"mutex", "dss"}. Default flipped to "dss".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
63 lines
2.1 KiB
Plaintext
63 lines
2.1 KiB
Plaintext
# GTA-UAV Balanced: Asymmetric DINOv3 (WEB+SAT) with L1/L2/L3 captions.
|
|
# Symmetric InfoNCE + MutuallyExclusiveSampler (no false negatives).
|
|
# 10 epochs, MONA all 24 blocks, 1024-dim retrieval, hard negative bank.
|
|
|
|
import src.losses.multi_infonce
|
|
|
|
# ---- Training ----
|
|
TrainConfigGTAUAV.epochs = 10
|
|
TrainConfigGTAUAV.batch_size = 8
|
|
TrainConfigGTAUAV.num_workers = 4
|
|
TrainConfigGTAUAV.learning_rate = 1e-4
|
|
TrainConfigGTAUAV.text_lr_factor = 0.1
|
|
TrainConfigGTAUAV.weight_decay = 1e-4
|
|
TrainConfigGTAUAV.grad_clip = 1.0
|
|
TrainConfigGTAUAV.grad_accum_steps = 8
|
|
TrainConfigGTAUAV.use_amp = True
|
|
TrainConfigGTAUAV.eval_every = 1
|
|
TrainConfigGTAUAV.warmup_epochs = 2
|
|
TrainConfigGTAUAV.seed = 42
|
|
TrainConfigGTAUAV.device = "cuda"
|
|
|
|
# ---- Model ----
|
|
TrainConfigGTAUAV.init_gate = 0.7
|
|
TrainConfigGTAUAV.baseline_mode = False
|
|
TrainConfigGTAUAV.shared_encoder = False
|
|
TrainConfigGTAUAV.gradient_checkpointing = True
|
|
|
|
# ---- Loss ----
|
|
TrainConfigGTAUAV.loss_type = "symmetric"
|
|
TrainConfigGTAUAV.tau_init = 0.07
|
|
TrainConfigGTAUAV.label_smoothing = 0.1
|
|
TrainConfigGTAUAV.learnable_temperature = True
|
|
TrainConfigGTAUAV.weight_q2g = 0.6
|
|
TrainConfigGTAUAV.weight_g2q = 0.4
|
|
TrainConfigGTAUAV.neg_bank_size = 4096
|
|
|
|
# ---- Sampling ----
|
|
TrainConfigGTAUAV.sampler_type = "dss" # "dss" or "mutex"
|
|
TrainConfigGTAUAV.dss_warmup_epochs = 1 # first N epochs use mutex-only (untrained embeds not useful)
|
|
TrainConfigGTAUAV.dss_reembed_every = 1
|
|
TrainConfigGTAUAV.use_mutex_sampler = True # legacy flag, kept True unless disabling both samplers
|
|
|
|
# ---- Output ----
|
|
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text"
|
|
|
|
# ---- Tracking ----
|
|
TrainConfigGTAUAV.use_wandb = False
|
|
TrainConfigGTAUAV.use_tb = True
|
|
TrainConfigGTAUAV.use_gradcam = True
|
|
TrainConfigGTAUAV.gradcam_every = 5
|
|
TrainConfigGTAUAV.use_profiler = False
|
|
TrainConfigGTAUAV.log_grad_norms = True
|
|
|
|
# ---- InfoNCELoss (gin-configurable) ----
|
|
InfoNCELoss.temperature_init = 0.07
|
|
InfoNCELoss.learnable_temperature = True
|
|
InfoNCELoss.label_smoothing = 0.1
|
|
InfoNCELoss.weight_q2g = 0.6
|
|
InfoNCELoss.weight_g2q = 0.4
|
|
InfoNCELoss.tau_min = 0.01
|
|
InfoNCELoss.tau_max = 0.1
|
|
InfoNCELoss.hard_mining_k = 512 # 0 = use whole queue (disable mining)
|