Add DynamicSimilaritySampler — embedding-kNN batches with mutex constraint
Batches assembled from visually-similar drone queries pressure the model to
learn finer discriminative features. Random mutex batches average ~0.26
pairwise cosine similarity in query embedding space; DSS batches average
~0.71 — confirming the lookalikes grouping works as intended.
Algorithm per batch:
1. Pick a random seed drone from the remaining pool.
2. Rank the entire remaining pool by cosine similarity to the seed.
3. Walk the ranking in descending order; add items whose sat_candidates
don't collide with the batch's already-claimed set.
4. Drop the seed if no valid batch can be assembled (rare mutex deadlock).
Inherits MutuallyExclusiveSampler semantics — no false negatives. Degrades
gracefully to mutex-only when no embeddings are set (warmup epochs, or if
`sampler_type="mutex"` is chosen).
Integration in `train_gtauav.py`:
- New `_embed_drone_queries` helper: model.encode_query forwarded over
GTAUAVDroneQuery, returns [N, D] CPU tensor. ~13s per 1024 queries on
a 4090 → ~5 min for the full 25K train set.
- Epoch loop re-embeds every `dss_reembed_every` epochs after a `dss_warmup_epochs`
warmup (first epochs use mutex-only since untrained embeddings aren't
informative for kNN).
- Config: `sampler_type` ∈ {"mutex", "dss"}. Default flipped to "dss".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -35,7 +35,10 @@ TrainConfigGTAUAV.weight_g2q = 0.4
|
||||
TrainConfigGTAUAV.neg_bank_size = 4096
|
||||
|
||||
# ---- Sampling ----
|
||||
TrainConfigGTAUAV.use_mutex_sampler = True
|
||||
TrainConfigGTAUAV.sampler_type = "dss" # "dss" or "mutex"
|
||||
TrainConfigGTAUAV.dss_warmup_epochs = 1 # first N epochs use mutex-only (untrained embeds not useful)
|
||||
TrainConfigGTAUAV.dss_reembed_every = 1
|
||||
TrainConfigGTAUAV.use_mutex_sampler = True # legacy flag, kept True unless disabling both samplers
|
||||
|
||||
# ---- Output ----
|
||||
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text"
|
||||
|
||||
Reference in New Issue
Block a user