Add DynamicSimilaritySampler — embedding-kNN batches with mutex constraint
Batches assembled from visually-similar drone queries pressure the model to
learn finer discriminative features. Random mutex batches average ~0.26
pairwise cosine similarity in query embedding space; DSS batches average
~0.71 — confirming the lookalikes grouping works as intended.
Algorithm per batch:
1. Pick a random seed drone from the remaining pool.
2. Rank the entire remaining pool by cosine similarity to the seed.
3. Walk the ranking in descending order; add items whose sat_candidates
don't collide with the batch's already-claimed set.
4. Drop the seed if no valid batch can be assembled (rare mutex deadlock).
Inherits MutuallyExclusiveSampler semantics — no false negatives. Degrades
gracefully to mutex-only when no embeddings are set (warmup epochs, or if
`sampler_type="mutex"` is chosen).
Integration in `train_gtauav.py`:
- New `_embed_drone_queries` helper: model.encode_query forwarded over
GTAUAVDroneQuery, returns [N, D] CPU tensor. ~13s per 1024 queries on
a 4090 → ~5 min for the full 25K train set.
- Epoch loop re-embeds every `dss_reembed_every` epochs after a `dss_warmup_epochs`
warmup (first epochs use mutex-only since untrained embeddings aren't
informative for kNN).
- Config: `sampler_type` ∈ {"mutex", "dss"}. Default flipped to "dss".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -101,6 +101,7 @@ Eval: Resize(256) + CenterCrop(256) + ImageNet normalization.
|
||||
| `src/models/asymmetric_encoder.py` | DINOv3ViT + TextFusionMLP + AsymmetricEncoder + GatedFusion + encode_query/encode_gallery (per-sample caption mask) |
|
||||
| `src/datasets/gtauav_dataset.py` | GTA-UAV-LR loader + L1/L2/L3 captions + GTAUAVSatGallery/GTAUAVDroneQuery (full retrieval eval) |
|
||||
| `src/datasets/mutually_exclusive_sampler.py` | BatchSampler: drone'ы в батче не делят sat_candidates (no false negatives) |
|
||||
| `src/datasets/dynamic_similarity_sampler.py` | DSS: embedding-kNN + mutex — батчи из визуально похожих drone'ов (hard negatives) |
|
||||
| `src/losses/multi_infonce.py` | **Primary:** SymmetricInfoNCE + MoCo queue, learnable τ clamp [0.01, 0.1], weights q2g=0.6 g2q=0.4, `hard_mining_k` для top-K hardest negatives |
|
||||
| `src/losses/weighted_infonce.py` | Alternative: per-sample adaptive label smoothing (активируется `loss_type="weighted"`) |
|
||||
| `src/losses/hard_negatives.py` | NegativeMemoryBank (MoCo-style FIFO queue 4096 × 1024) |
|
||||
@@ -213,7 +214,7 @@ Meta-файл `meta/seg_filter.json`: исключение изображени
|
||||
- **Scheduler:** linear warmup (2 epochs) + cosine annealing (per-step)
|
||||
- **Loss:** SymmetricInfoNCE (q2g=0.6, g2q=0.4) с learnable τ (init=0.07, clamp [0.01, 0.1])
|
||||
- **Hard mining:** top-K=512 hardest negatives per query из MoCo queue (размер 4096); `hard_mining_k=0` отключает
|
||||
- **Batch sampler:** MutuallyExclusiveSampler — batches disjoint по sat_candidates (на bs=8 сохраняет 100% entries)
|
||||
- **Batch sampler:** `sampler_type="dss"` (default) — DynamicSimilaritySampler с re-embedding каждую эпоху: пакует визуально похожих drone'ов в один батч (+hardness) с mutex-constraint (no false negatives). Первая эпоха warmup mutex-only. Средний in-batch cosine sim ~0.71 vs 0.26 у mutex. `sampler_type="mutex"` — без DSS
|
||||
- **Eval:** full satellite gallery (~2684 unique tiles для test_20) с multi-match R@K (учитывает все positive/semi-positive)
|
||||
- **Augmentations:**
|
||||
- Drone: RandomResizedCrop(0.7-1.0), HFlip, Rotation(15°), ColorJitter, Grayscale(5%), GaussianBlur
|
||||
|
||||
Reference in New Issue
Block a user