fuse_proj: Initial operational package for 3 researchers (Pavlenko/Blizno/Moroz)

Multimodal fusion research on StripNet+GTA-UAV proxy: - 3 independent fusion tracks: condition-aware (A), token/bottleneck (B), role-aware (C) - Shared interfaces, protocol, dataset audit, baseline benchmarks - Canonical version-chain references to vault (SPEC, ANALYSIS, TRIAGE) - Personalized task plans and decision tables for each researcher - 3 generated DOCX task assignment files with milestones and DoD checklist - Full modality dropout diagnostics and missing-modality robustness requirements - Data contract, benchmark registry, experiment tracking infrastructure Operational documents: - docs/00_project/: MERIDIAN context, protocol, repository reuse guide, experiment specification - docs/01_tasks/: Master assignment + 3 individual researcher tracks + joint integration - docs/02_references/: Core literature, version-chain bases, code maps - docs/03_codebase_guides/: Existing code snapshots from vault - scripts/: gen_task_plans.js (DOCX generation), placeholder infrastructure - vendor_reference/: Snapshots of caption_test, depth_edges_annotate, existing SOFIA/SegModel code - reports/, results/, experiments/: Shared output structure for all 3 researchers 3 DOCX files generated from gen_task_plans.js (Times New Roman 14pt, GOST format): - План_заданий_Павленко_БВ.docx (Condition-Aware track, fusion API owner) - План_заданий_Близно_МВ.docx (Token/Bottleneck track, benchmark owner) - План_заданий_Мороз_ЕС.docx (Role-Aware track, data contract owner) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-11 17:16:57 +03:00
commit 2c6a00a4ca
155 changed files with 39765 additions and 0 deletions
--- a/vendor_reference/caption_test/CLAUDE.md
+++ b/vendor_reference/caption_test/CLAUDE.md
@@ -0,0 +1,447 @@
+# Caption Quality Test for Cross-View Geo-Localization
+
+## Архитектура системы (v3, 2026-04-24) — GTA-UAV эксперимент
+
+```
+Shared DINOv3 ViT-L/16 (LVD-1689M, frozen + MONA in last 12/24 blocks)
+для обеих веток — drone и satellite кодируются одним encoder.
+
+QUERY BRANCH (drone + L1/L2/L3 captions):
+  drone_img [B,3,256,256] --> DINOv3 ViT-L/16 (shared) --> d_img [B,1024]
+                                                                        |
+  L1 --> DGTRS-CLIP (248 tok) --> z₁ [768] --\                         |
+  L2 --> DGTRS-CLIP (248 tok) --> z₂ [768] ---+-- cat --> MLP(2304→1024→1024) --> d_txt [B,1024]
+  L3 --> DGTRS-CLIP (248 tok) --> z₃ [768] --/                         |
+                                                                        |
+                    q = σ(α_q)·d_img + (1−σ(α_q))·d_txt   GatedFusion_q
+                                        |
+                                  q̂ = q/‖q‖₂ --> query [B,1024]
+
+GALLERY BRANCH (satellite + satellite captions):
+  sat_img [B,3,256,256] --> DINOv3 ViT-L/16 (shared, same weights) --> s_img [B,1024]
+                                                                      |
+  sat_L1 --> DGTRS-CLIP --> z₁ --\                                    |
+  sat_L2 --> DGTRS-CLIP --> z₂ ---+-- cat --> MLP (shared) --> s_txt [B,1024]
+  sat_L3 --> DGTRS-CLIP --> z₃ --/                                    |
+                                                                      |
+                    g = σ(α_g)·s_img + (1−σ(α_g))·s_txt   GatedFusion_g
+                                        |
+                                  ĝ = g/‖g‖₂ --> gallery [B,1024]
+
+Retrieval space: 1024-dim (DINOv3 native, без projection layers)
+TextFusionMLP shared между query и gallery (одинаковый формат captions)
+Для sat images без captions: s_txt=None → g = s_img (gate passthrough)
+
+LOSS: L = 0.6·CE(q̂·ĝᵀ/τ, targets) + 0.4·CE(ĝ·q̂ᵀ/τ, targets)
+      τ = 1/exp(logit_scale), learnable, clamped [0.01, 0.1], init=0.07
+      label_smoothing=0.1
+
+BATCH SAMPLING: MutuallyExclusiveSampler — в одном батче нет двух drone'ов
+      с пересекающимися sat_candidates (исключает false negatives, которые
+      иначе появляются из-за multi-positive структуры GTA-UAV).
+
+BASELINE: σ(α_q)=σ(α_g)=1.0, text disabled, DGTRS not loaded
+```
+
+### Text hierarchy (L1/L2/L3)
+- **L1 overview:** первое предложение P1 — краткое описание land-cover (15-30 tok)
+- **L2 full:** полные P1 + P2 — inventory + spatial layout (100-200 tok)
+- **L3 fingerprint:** P3 — уникальные landmarks для matching (20-50 tok)
+- **Fusion:** z_text = MLP([z₁; z₂; z₃]) — concat 3×768 → Linear(2304,1024) → GELU → Linear(1024,1024)
+- **Shared MLP** между query и gallery ветками (одинаковый формат captions)
+- **Satellite captions:** 6,546 из 14,640 sat images имеют captions. Для остальных gate passthrough (g = s_img) — **per-sample mask** в `_fuse_with_mask` возвращает чистые image features для samples без caption (без шума от пустых строк)
+
+### Text encoder: DGTRS-CLIP (official architecture)
+- Код: `src/models/dgtrs/` — из github.com/MitsuiChen14/DGTRS (Apache-2.0)
+- KPS positional embedding: mask1 (pos 0-19, frozen) + mask2 (pos 20-247, trainable)
+- Transformer: sequence-first (LND), nn.MultiheadAttention, 12 layers
+- Tokenizer: BPE SimpleTokenizer (248 tokens, vocab 49408)
+
+### Trainable parameters: 7.06M из 434M (1.63%)
+- **MONA adapters** (shared DINOv3): 3.5M (2 per block × 12 last blocks, bottleneck=64)
+- **LoRA** (DGTRS-CLIP): 147K (Q+V, rank=4, 12 blocks)
+- TextFusionMLP (shared): Linear(2304,1024)+GELU+Linear(1024,1024) = ~3.4M
+- gate α_q + α_g: 2 scalars
+- logit_scale: 1 scalar (learnable temperature)
+- DINOv3 (1 encoder) + DGTRS: frozen backbone weights
+- **Без projection layers** — retrieval space = DINOv3 native 1024-dim
+- **AMP:** frozen layers fp16, adapters + loss fp32
+- **Примечание:** ранее была asymmetric setup (2×DINOv3 WEB+SAT, MONA во всех 24 блоках) с 17.6M trainable / 733M total. Упростили до shared + last-12 MONA.
+
+### Optimizer & Scheduler
+- **AdamW** с per-group LR: projections lr=1e-4, text encoder lr=1e-5
+- **Linear warmup** (2 epochs) + **cosine annealing** (per-step)
+- **Gradient clipping:** max_norm=1.0
+- **AMP:** fp16 для model forward, fp32 для loss (learnable temperature overflow fix)
+
+### Image input: 256x256
+DINOv3 ViT-L/16 с patch_size=16 → 16x16=256 patches на 256x256.
+Train: augmentations (drone: crop+flip+rot+jitter+blur, sat: crop+flip+jitter).
+Eval: Resize(256) + CenterCrop(256) + ImageNet normalization.
+
+### Предыдущая архитектура (v2) — UAV-GeoLoc эксперимент
+
+Использовала GeoRSCLIP ViT-B/32 (512-dim) для обеих веток + template captions.
+Код в `src/models/dual_encoder.py`, `src/datasets/visloc_with_captions.py`.
+
+## Ключевые файлы
+
+### V2 (UAV-GeoLoc, GeoRSCLIP)
+| Файл | Назначение |
+|------|-----------|
+| `src/models/dual_encoder.py` | GeoRSCLIP + GatedFusion + projection heads |
+| `src/losses/multi_infonce.py` | InfoNCE с cosine temperature schedule |
+| `src/datasets/visloc_with_captions.py` | UAV-GeoLoc loader + template captions из path metadata |
+| `src/training/train.py` | Training loop, логирование loss/gate/tau |
+| `src/eval/evaluate.py` | R@K metrics, delta_r_at_1 |
+| `scripts/compare_runs.py` | Markdown/JSON сравнение baseline vs caption runs |
+| `scripts/generate_captions.py` | Offline caption generation (template/VLM/hybrid) |
+
+### V3 (GTA-UAV, DINOv3 + DGTRS-CLIP) — DONE
+| Файл | Назначение |
+|------|-----------|
+| `src/models/dgtrs/model.py` | Официальная архитектура DGTRS-CLIP text encoder (Apache-2.0) |
+| `src/models/dgtrs/simple_tokenizer.py` | BPE tokenizer (248 tokens, vocab 49408) |
+| `src/models/asymmetric_encoder.py` | DINOv3ViT + TextFusionMLP + AsymmetricEncoder + GatedFusion + encode_query/encode_gallery (per-sample caption mask) |
+| `src/datasets/gtauav_dataset.py` | GTA-UAV-LR loader + L1/L2/L3 captions + GTAUAVSatGallery/GTAUAVDroneQuery (full retrieval eval) |
+| `src/datasets/mutually_exclusive_sampler.py` | BatchSampler: drone'ы в батче не делят sat_candidates (no false negatives) |
+| `src/datasets/dynamic_similarity_sampler.py` | DSS: embedding-kNN + mutex — батчи из визуально похожих drone'ов (GPU/CPU, опциональный LSH) |
+| `src/datasets/lsh_index.py` | Random-projection cosine-LSH для approximate kNN (opt-in; `dss_use_lsh=True`) |
+| `src/datasets/embedding_cache.py` | Дисковый кеш для drone embeddings — skip re-embed на resume |
+| `src/losses/multi_infonce.py` | **Primary:** SymmetricInfoNCE + MoCo queue, learnable τ clamp [0.01, 0.1], weights q2g=0.6 g2q=0.4, `hard_mining_k` для top-K hardest negatives |
+| `src/losses/weighted_infonce.py` | Alternative: per-sample adaptive label smoothing (активируется `loss_type="weighted"`) |
+| `src/losses/hard_negatives.py` | NegativeMemoryBank (MoCo-style FIFO queue 4096 × 1024) |
+| `src/training/train_gtauav.py` | Training loop: full-gallery `_evaluate`, mutex sampler wiring, loss_type switch |
+| `scripts/smoke_eval.py` / `scripts/smoke_train.py` | Регрессионные smoke-тесты для eval и train pipeline |
+| `src/training/trackers.py` | Unified experiment tracker: W&B + TensorBoard + CSV |
+| `src/training/grad_monitor.py` | Gradient norm monitoring per param group |
+| `src/training/gradcam.py` | Grad-CAM visualization для DINOv3 encoders |
+| `src/training/profiling.py` | PyTorch Profiler wrapper + torchinfo model summary |
+| `src/training/plot_metrics.py` | Seaborn/matplotlib plots (каждую эпоху) |
+| `conf/gtauav_balanced.gin` | Shared encoder, MONA 12/24, with text, gate=0.7, 10 epochs |
+| `conf/gtauav_baseline.gin` | Shared encoder, MONA 12/24, no text, gate=1.0 |
+| `conf/gtauav_balanced_asym.gin` | Asymmetric (WEB+SAT), MONA 24/24, with text — overrides gtauav_balanced.gin |
+| `conf/gtauav_baseline_asym.gin` | Asymmetric (WEB+SAT), MONA 24/24, no text |
+| `conf/gtauav_text_heavy.gin` | Text-heavy, gate=0.3 |
+| `conf/gtauav_image_heavy.gin` | Image-heavy, gate=0.9 |
+| `scripts/make_split.py` | 80/20 random split из всех пар |
+| `scripts/filter_segmentation.py` | Scan segm masks, output meta JSON (exclude >=90% bg+water) |
+
+## Backbones (v3)
+
+### DINOv3 ViT-L/16 — Shared (web pretrained)
+- **Checkpoint:** `nn_models/DINO_WEB/dinov3-vitl16-pretrain-lvd1689m.pth`
+- **Arch:** ViT-L/16, 24 layers, 16 heads, hidden=1024, MLP=4096, 303M params
+- **Input:** 256x256, ImageNet normalization, patch=16 → 256 patches
+- **Register tokens:** 4, RoPE theta=100.0
+- **MONA:** 24 адаптера в последних 12 блоках (blocks 12-23), bottleneck=64, 3.5M trainable
+- **Status:** frozen кроме MONA
+- **Примечание:** ранее asymmetric — использовался отдельно `nn_models/DINO_SAT/model.safetensors` (sat493m pretrain) для satellite ветки. Упростили до shared WEB-энкодера.
+
+### DGTRS-CLIP ViT-L-14 (LRSCLIP) — Text encoder
+- **Checkpoint:** `nn_models/LRSCLIP/DGTRS-CLIP-ViT-L-14.pt`
+- **Код:** `src/models/dgtrs/` — официальная архитектура из github.com/MitsuiChen14/DGTRS
+- **Text dim:** 768, max tokens: 248 (KPS: mask1 pos 0-19 frozen + mask2 pos 20-247 trainable)
+- **Transformer:** 12 layers, 12 heads, sequence-first (LND), QuickGELU
+- **Tokenizer:** BPE SimpleTokenizer (vocab 49408), 248 token context
+- **Содержит:** полную CLIP модель (visual + text), используем только text encoder (124M params)
+- **Status:** partial unfreeze (last resblock + ln_final + text_projection, ~7.6M trainable)
+
+### GeoRSCLIP ViT-B/32 (v2, legacy)
+- **Checkpoint:** `checkpoints/RS5M_ViT-B-32.pt`
+- **Image encoder:** ViT-B/32, 224x224, 512-dim, ~86M params — frozen
+- **Text encoder:** 77 tokens, 512-dim — partial unfreeze
+
+## GatedFusion
+
+- `query = sigma(alpha) * drone_feat + (1 - sigma(alpha)) * text_feat`
+- `alpha` — один learnable scalar в logit-space
+- `init_gate = 0.7` → начальный вес image = 70%, text = 30%
+- `baseline_mode = True` → gate = 1.0, text полностью игнорируется
+- Gate value логируется каждую эпоху для интерпретации вклада текста
+
+## Text Hierarchy (L1/L2/L3)
+
+Три уровня описаний из VLM-generated captions:
+
+| Уровень | Контент | Длина | Источник |
+|---|---|---|---|
+| L1 overview | Краткое описание сцены | <=30 tok | Конденсация P1 |
+| L2 full description | Детальное описание через Qwen3-VL | <=200 tok | Полный P1+P2 |
+| L3 fingerprint | Ключевые landmark'ы | <=30 tok | Конденсация P3 |
+
+Все три уровня кодируются одним LRSCLIP (248 tok max).
+Альтернатива (Stage 2): RemoteCLIP для L1/L3 + LRSCLIP для L2.
+
+## Датасет: GTA-UAV-LR
+
+- **RGB:** `/home/servml/Документы/datasets/GTA-UAV-LR/`
+  - Drone: 33,763 PNG (512x384), altitudes 100-600m
+  - Satellite: 14,640 PNG (256x256 RGBA)
+  - Pairs: `cross-area-drone2sate-{train,test}.json` (primary split)
+  - Metadata: `*_drone_meta.csv` (height, yaw, roll, pitch)
+  - Origin: GTA V simulation (Los Santos)
+
+- **Captions:** `/home/servml/Документы/datasets/GTA-UAV-LR-captions/`
+  - Drone: 33,411 JSON (32,635 multi-paragraph P1/P2/P3 + 776 short water-only)
+  - Satellite: 6,546 JSON (все multi-paragraph)
+  - Формат: 3 абзаца (P1 Inventory + P2 Spatial + P3 Fingerprint)
+  - Token counts: ~430 output tokens per caption
+
+- **Segmentation:** `/home/servml/Документы/datasets/GTA-UAV-LR-aug/`
+  - 48,403 images, 17 классов (background, building, road, vegetation, water, ...)
+  - Modalities: segm/, depth/, edge/, chm/, safetensors/
+  - Query: 512x512, DB: 256x256
+
+### Фильтрация сегментации
+Meta-файл `meta/seg_filter.json`: исключение изображений с >=90% background(class 0) + water(class 4).
+- **Total:** 48,403 → **Passed:** 37,498 (77.5%) / **Excluded:** 10,905 (22.5%)
+- Drone: 31,188 passed / 2,575 excluded
+- Satellite: 6,310 passed / 8,330 excluded (преимущественно open water tiles)
+
+## Датасет: UAV-GeoLoc (v2, legacy)
+
+- **Путь:** `/mnt/data1tb/cvgl_datasets/UAV-GeoLoc/`
+- **Train:** 206,108 queries, 94,709 DB crops (140 scenes, Terrain split)
+- **Val:** 62,368 queries, 26,597 DB crops (40 scenes)
+- **Test:** 33,472 queries, 11,684 DB crops (20 scenes)
+- **Index:** `Index/train_query.txt` — `query_path 0 pos_crop1 pos_crop2 ...`
+
+## Конфигурации
+
+### V3 (GTA-UAV)
+Параметры:
+- 10 epochs, batch 64, AMP, image 256x256
+- **Optimizer:** AdamW, per-group LR: proj=1e-4, text=1e-5 (10x lower)
+- **Scheduler:** linear warmup (2 epochs) + cosine annealing (per-step)
+- **Loss:** SymmetricInfoNCE (q2g=0.6, g2q=0.4) с learnable τ (init=0.07, clamp [0.01, 0.1])
+- **Hard mining:** top-K=512 hardest negatives per query из MoCo queue (размер 4096); `hard_mining_k=0` отключает
+- **Batch sampler:** `sampler_type="dss"` (default) — DynamicSimilaritySampler с re-embedding каждую эпоху: пакует визуально похожих drone'ов в один батч (+hardness) с mutex-constraint (no false negatives). Первая эпоха warmup mutex-only. Средний in-batch cosine sim ~0.71 vs 0.26 у mutex. kNN на GPU (`dss_knn_device="cuda"`) — 1.6s vs 17s на CPU. Опциональный LSH (`dss_use_lsh=True`) для scale 100K+. Embedding cache (`dss_cache_dir`) — skip re-embed на resume.
+- **Eval:** full satellite gallery (~2684 unique tiles для test_20) с multi-match R@K (учитывает все positive/semi-positive)
+- **Augmentations:**
+  - Drone: RandomResizedCrop(0.7-1.0), HFlip, Rotation(15°), ColorJitter, Grayscale(5%), GaussianBlur
+  - Satellite: RandomResizedCrop(0.7-1.0), HFlip, ColorJitter, Grayscale(5%)
+  - Eval: Resize+CenterCrop (clean, no augmentation)
+- **Split:** 80/20 random из всех 33,708 пар (`meta/train_80.json` / `meta/test_20.json`)
+- Train: 26,966 → 24,891 after seg filter
+- Test: 6,742 → 6,252 after seg filter
+- Скрипт: `python -m scripts.make_split --ratio 0.8 --seed 42`
+
+### V3 (GTA-UAV, gin)
+| Конфиг | Gate init | Описание |
+|--------|-----------|----------|
+| `conf/gtauav_balanced.gin` | 0.7 (30% text) | **Primary test** — shared DINOv3 WEB, MONA 12/24 |
+| `conf/gtauav_baseline.gin` | 1.0 (no text) | Reference baseline (shared, MONA 12/24) |
+| `conf/gtauav_balanced_asym.gin` | 0.7 (30% text) | Asymmetric (WEB+SAT), MONA 24/24 — full-capacity variant |
+| `conf/gtauav_baseline_asym.gin` | 1.0 (no text) | Asymmetric baseline for Δ R@1 vs balanced_asym |
+| `conf/gtauav_text_heavy.gin` | 0.3 (70% text) | Stress test |
+| `conf/gtauav_image_heavy.gin` | 0.9 (10% text) | Image-dominant |
+
+### V2 (UAV-GeoLoc, gin)
+| Конфиг | Gate init | Описание |
+|--------|-----------|----------|
+| `conf/balanced.gin` | 0.7 (30% text) | **Primary test** |
+| `conf/baseline_no_text.gin` | 1.0 (no text) | Reference baseline |
+| `conf/text_heavy.gin` | 0.3 (70% text) | Stress test |
+
+## Запуск
+
+### V3 (GTA-UAV)
+```bash
+# 1. Filter segmentation (exclude 90%+ background/water)
+python -m scripts.filter_segmentation --output meta/seg_filter.json
+
+# 2. Train with gin config (recommended)
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json
+
+# 3. Baseline (no text)
+python -m src.training.train_gtauav --config conf/gtauav_baseline.gin \
+    --filter-meta meta/seg_filter.json
+
+# 4. With diagnostics (W&B + Grad-CAM + Profiler)
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json --wandb --gradcam --profile
+
+# 5. CLI overrides (gin params take priority)
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json \
+    --gin-param 'TrainConfigGTAUAV.batch_size=16'
+
+# 6. Compare
+python -m scripts.compare_runs \
+    --baseline_report out/gtauav/baseline/eval_report.json \
+    --full_report out/gtauav/with_text/eval_report.json \
+    --output out/gtauav/comparison.md
+
+# 7. TensorBoard
+tensorboard --logdir out/gtauav/with_text/tb_logs
+```
+
+### V2 (UAV-GeoLoc)
+```bash
+python -m src.training.train --config conf/baseline_no_text.gin
+python -m src.training.train --config conf/balanced.gin
+python -m scripts.compare_runs \
+    --baseline_report out/caption_test/baseline_no_text/eval_report.json \
+    --full_report out/caption_test/balanced/eval_report.json \
+    --output out/caption_test/comparison.md
+```
+
+## Метрики и Decision rule
+
+**Primary metric:** Delta R@1 (drone -> satellite)
+
+| Delta R@1 | Verdict |
+|-----------|---------|
+| >= +3% | PASS — captions informative, proceed to production |
+| +1% to +3% | MARGINAL — add VLM refinement, re-run |
+| 0 to +1% | WEAK — redesign caption pipeline |
+| < 0 | HARMFUL — critical bug |
+
+**Eval metrics:** R@1, R@5, R@10 + AP (MRR) для обоих направлений: q2g (drone→satellite) и g2q (satellite→drone). g2q считается через инвертированный GT (для каждого sat-tile собираются drone-индексы из `valid_idx_per_query`); знаменатель — sat-tiles, у которых есть хотя бы один positive drone в (под)выборке.
+**Splits (GTA-UAV):** cross-area (primary, harder) и same-area (sanity check)
+**Logged per epoch:** loss, temperature (tau), gate value (sigma(alpha)), lr
+
+## Бюджет времени (RTX 4090, 24 GB)
+
+### V3 (GTA-UAV, DINOv3 ViT-L/16, 256x256)
+| Фаза | Оценка |
+|------|--------|
+| VRAM: DINOv3-L (shared) + LRSCLIP + batch 64 | ~10-14 GB (было ~18-22 с 2× DINOv3) |
+| GPU mem (smoke test, batch 4) | 3.1 GB |
+| Batch size | 64 (default) |
+| Total params | 434M (7.06M trainable, 1.63%) — shared encoder + MONA в last 12/24 blocks |
+
+### V2 (UAV-GeoLoc, GeoRSCLIP)
+| Фаза | Время |
+|------|-------|
+| Один training run (10 epochs, 206K queries, batch 128) | ~15-30 мин |
+| Full test (3 варианта) | ~1-1.5 ч |
+| Evaluation per run | ~2-5 мин |
+
+## Связанные проекты
+
+### Text Annotation Pipeline
+- **Путь:** `/home/servml/Документы/pikaliov_obsidian/(Полякова ВЕ_Система для генерации текстовых описаний для БПЛА)/2_work/2_text_annotation/code/`
+- **VLM:** Qwen3-VL-8B AWQ (1.68 s/img)
+- **Scoring:** SigLIP 2 (drone P3) + CLIP-RSICD (satellite P1+P2)
+- **Формат описаний:** 3 абзаца (P1 Inventory + P2 Spatial Map + P3 Fingerprint)
+- **Метрики:** FDR, FNR, NumAcc, LLaVA-Critic C1-C6
+
+### UAV-VisLoc Prepare
+- **Путь:** `/home/servml/Документы/code/Yaroslav/UAV-VisLoc-prepare/scripts/prepare_dataset.py`
+- **Статус:** выполнен (2026-04-17), данные в `/home/servml/Документы/datasets/UAV_VisLoc_processed/` (25 GB)
+- **Задача:** нарезка satellite кропов 512x512, stride 256 + resize drone -> 512x512
+- **Подробности:** см. ниже
+
+---
+
+## Downstream: NADEZHDA Teacher (DINOv3 + Multi-FiLM)
+
+caption-test — первый этап валидации. При Δ R@1 >= +3% переход к полному teacher'у:
+
+```
+Этап 1 (caption-test): GeoRSCLIP + GatedFusion(text) → валидация текста
+Этап 2 (teacher):      DINOv3-L + Multi-FiLM(depth, seg, CHM, normals, text)
+Этап 3 (distillation): teacher ~300M → student ~5M → Jetson Orin NX
+```
+
+### Auxiliary modalities (предвычисляются из 512x512 офлайн)
+
+| Модальность | Модель | Формат для teacher | Каналы |
+|---|---|---|---|
+| Depth | DepthAnything V2 | continuous, log(1+d) | 1 |
+| Normals | Sobel от depth | continuous | 3 |
+| Segmentation | SegFormer-B5 | binary per-class masks (top-K) | 16-17 |
+| Canopy Height | Meta HRCH | binary bins (1-5m, 5-15m, >15m) + occupancy | 4-5 |
+| Text | Qwen3-VL-8B / MobileCLIP2 | embedding | - |
+
+### Асимметрия sat/drone
+- CHM: только satellite (модель обучена на nadir, на oblique drone не работает)
+- Satellite: ~27 aux каналов, Drone: ~21 aux каналов
+
+### Fusion: Multi-FiLM
+```
+aux_features → FiLM(γ, β) → γ * DINOv3_tokens + β
+```
+Binary masks — natural FiLM gates. Modality dropout: text 0.3, CHM 0.5, seg 0.15, depth 0.1.
+
+### Планируемый эксперимент H5
+Сравнение T_bin (binary masks) vs T_pyr (native feature pyramids) vs T_hybrid.
+Прогноз: binary masks лучше на cross-domain из-за робастности к aux-model artifacts.
+
+---
+
+## Датасеты (справочник)
+
+### UAV-VisLoc
+- **Путь:** `/home/servml/Документы/datasets/UAV_VisLoc_dataset/`
+- **Структура:** 11 маршрутов (папки `01`-`11`), каждая содержит:
+  - `drone/` — drone-снимки (`XX_NNNN.JPG`)
+  - `satelliteXX.tif` — спутниковая карта
+  - `XX.csv` — GPS-метаданные drone (num, filename, date, lat, lon, height, Omega, Kappa, Phi1, Phi2)
+- **Исключение:** маршрут `09` — спутник разбит на 4 тайла (`satellite09_01-01.tif` и т.д.)
+- **Satellite coordinates:** `satellite_ coordinates_range.csv` — bbox каждой карты (LT_lat_map, LT_lon_map, RB_lat_map, RB_lon_map)
+- **Splits:** `visloc_train.csv`, `visloc_test.csv` — списки drone-снимков (TSV, full absolute paths)
+- **Размеры:** Drone 3976x2652 / 3000x2000 (6774 снимков), Satellite от 3000x170 до 43421x38408
+- **GSD спутника:** ~0.30 м/px (единый zoom level Google Earth для всех карт). GSD по долготе варьируется 0.23-0.27 м/px из-за косинусного эффекта широты (40°N vs 25°N), но это не разная высота съёмки. Кроп 512x512 покрывает ~154x154 м везде.
+
+### UAV-GeoLoc
+- **Путь:** `/mnt/data1tb/cvgl_datasets/UAV-GeoLoc/`
+- **Подмножества:** Country (171 scene), Terrain (200 scenes), Rot (1 scene)
+- **Формат пар:** `positive.json`, `semi_positive.json`, `db_postion.txt`
+- **Index:** `train_query.txt` — `query_path label pos_crop1 pos_crop2 ...`
+- **Drone:** синтетика Google Earth Studio 3D, 512x512, FOV 30deg, heights 100/125/150m
+- **Satellite:** кропы из merge.tif, доминирующий размер 200x200
+
+## Скрипт подготовки UAV-VisLoc
+
+- **Путь:** `/home/servml/Документы/code/Yaroslav/UAV-VisLoc-prepare/scripts/prepare_dataset.py`
+- **Статус:** выполнен (2026-04-17)
+
+### Запуск
+```bash
+python scripts/prepare_dataset.py \
+    --src /home/servml/Документы/datasets/UAV_VisLoc_dataset \
+    --dst /home/servml/Документы/datasets/UAV_VisLoc_processed \
+    --crop-size 512 --stride 256 --target-size 512
+```
+
+### Pipeline
+1. Resize drone -> 512x512 (JPEG, quality=95)
+2. Stitch satellite tiles для маршрута 09 (4 тайла -> 44800x33280)
+3. Нарезка satellite -> кропы 512x512, stride 256, сохранение без downscale (PNG)
+4. GPS для каждого кропа из bbox карты + позиция в grid
+5. Match drone->crop через vectorized haversine (positive = ближайший, semi-positive = +/-1 в grid)
+6. Metadata: `positive.json`, `semi_positive.json`, `db_postion.txt` (per route)
+7. Index: `train_query.txt`, `test_query.txt`, `train_db.txt`, `test_db.txt`, `all_db.txt`
+
+### Форматы выходных файлов (совместимость с UAV-GeoLoc)
+| Файл | Формат |
+|------|--------|
+| `positive.json` | `{frame_id: [crop_name]}`, ключ = frame ID без route prefix (`"0001"`) |
+| `semi_positive.json` | `{frame_id: [crop1, crop2, ...]}`, соседи +/-1 в grid |
+| `db_postion.txt` | tab-separated: `name\tlon\tlat\tscale_lon\tscale_lat` |
+| `train_query.txt` | `route/drone/file.JPG 0 route/DB/img/crop1.png ...` |
+| `train_db.txt` / `test_db.txt` | все кропы всех маршрутов (gallery одинаковая, split по query) |
+
+### Результаты (target-size 512)
+- Drone: 6,744 images 512x512 (без маршрута 07: 30 excluded)
+- Satellite кропов: 74,807 (512x512, без downscale)
+- Размер на диске: 25 GB
+- Median distance drone->crop: 25.9m, P99: 45.7m
+- Память при генерации: до ~8.7 GB RAM (маршрут 09 stitched 44800x33280)
+- Разрешение 512 для downstream задач (сегментация, depth, normals); resize до 256x256 в dataloader
+
+### Ревью и исправления (2026-04-17)
+1. `train_db.txt`/`test_db.txt` содержали только matched кропы -> теперь все ~74K (полная gallery)
+2. `db_position.txt` -> `db_postion.txt` (совместимость с UAV-GeoLoc), добавлены scale_lon/scale_lat, tab-separator
+3. `positive.json` ключи были filename (`01_0001.JPG`) -> теперь frame_id (`0001`)
+4. Semi-positive поиск O(n) -> O(1) через dict lookup по (x,y) grid
+5. Удален мёртвый код (`haversine_m`, `defaultdict` import)
+
+### Известные ограничения
+- Нет val split (только train/test, как в оригинальном UAV-VisLoc)
+- 6 drone в маршруте 06 (06_0093-06_0098) за пределами спутниковой карты (distance >1000m)
+- Большие спутниковые карты загружаются целиком в RAM при генерации (до 8.7 GB для route 09)
--- a/vendor_reference/caption_test/README.md
+++ b/vendor_reference/caption_test/README.md
@@ -0,0 +1,586 @@
+# Caption Quality Test for Cross-View Geo-Localization
+
+Validate whether generated text captions improve retrieval R@1 in cross-view
+geo-localization (drone-to-satellite).
+
+## Architecture
+
+### Overview
+
+Shared DINOv3 WEB encoder with MONA adapters (last 12 blocks, bfloat16),
+DGTRS-CLIP text fusion, and projection to 512-dim retrieval space.
+
+```
+┌─────────────────────────────── QUERY BRANCH ─────────────────────────────────┐
+│                                                                               │
+│  drone_img ──► DINOv3 ViT-L/16 WEB (frozen, 303M) ──► CLS [B,1024]         │
+│  [B,3,256,256]     + MONA adapters (3.5M, bf16)              │                │
+│                    (2 per block × last 12 of 24, bn=64)      │                │
+│                    + gradient checkpointing              Projection           │
+│                                                     Linear(1024→512)         │
+│                                                         d_img [B,512]        │
+│                                                              │                │
+│  L1 (overview)    ──► DGTRS-CLIP ViT-L-14 ──► z₁ [B,768] ─┐                 │
+│  L2 (full desc)   ──►   (frozen, 124M)    ──► z₂ [B,768] ─┼─ cat ──[B,2304] │
+│  L3 (fingerprint) ──►   + LoRA r=4 (147K) ──► z₃ [B,768] ─┘    │            │
+│                        (248 tok, KPS pos.)      TextFusionMLP (shared, 1.5M) │
+│                        + gradient checkpointing Linear(2304→512)→GELU→       │
+│                                                 Linear(512→512)              │
+│                                                       │                      │
+│                                                  d_txt [B,512]               │
+│                                                       │                      │
+│              q = σ(α_q)·d_img + (1−σ(α_q))·d_txt    GatedFusion_q           │
+│                            │                                                  │
+│                      q̂ = q / ‖q‖₂  ──► query [B,512]                         │
+└───────────────────────────────────────────────────────────────────────────────┘
+
+┌────────────────────────────── GALLERY BRANCH ────────────────────────────────┐
+│                                                                               │
+│  sat_img ──► DINOv3 WEB (shared encoder) ──► CLS ──► Projection              │
+│  [B,3,256,256]  + MONA (shared adapters)        s_img [B,512]                │
+│                                                      │                        │
+│  sat_L1 ──► DGTRS-CLIP ──► z₁ ─┐                                             │
+│  sat_L2 ──► (shared)    ──► z₂ ─┼─ cat ──► TextFusionMLP ──► s_txt [B,512]  │
+│  sat_L3 ──► + LoRA      ──► z₃ ─┘                 │                          │
+│                                                    │                          │
+│         g = σ(α_g)·s_img + (1−σ(α_g))·s_txt    GatedFusion_g                │
+│                            │                                                  │
+│                     ĝ = g / ‖g‖₂  ──► gallery [B,512]                        │
+└───────────────────────────────────────────────────────────────────────────────┘
+
+┌────────────────────────────── RETRIEVAL ──────────────────────────────────────┐
+│                                                                               │
+│  similarity = q̂ · ĝᵀ / τ      (τ learnable, init=0.07, clamp [0.01, 0.5])   │
+│  loss = 0.6·CE(q→g) + 0.4·CE(g→q)     (label_smoothing=0.1)                 │
+│                                                                               │
+│  Retrieval space: 512-dim (DINOv3 1024 → projection → 512)                   │
+│  All shared: one DINOv3, one MONA set, one DGTRS-CLIP, one TextFusionMLP     │
+│  For sat images without captions: s_txt=None → g = s_img (gate passthrough)   │
+│  BASELINE: σ(α) = 1.0 for both branches (text disabled, DGTRS not loaded)    │
+└───────────────────────────────────────────────────────────────────────────────┘
+
+┌──────────────────────────── DIAGNOSTICS PIPELINE ────────────────────────────┐
+│                                                                               │
+│  Per-batch ──► CSV (train_batches.csv)  ──► TensorBoard / W&B scalars        │
+│  Per-epoch ──► CSV (train.csv, val.csv) ──► Seaborn plots (PNG)              │
+│  Every N epochs ──► Grad-CAM heatmaps (drone + satellite DINOv3 last block)  │
+│  Epoch 0 (opt) ──► PyTorch Profiler (Chrome trace + CUDA timeline)           │
+│  Per-50-batch ──► Gradient norms per group (MONA, LoRA, MLP, gates, τ)       │
+│  On init ──► torchinfo model summary (model_summary.txt)                     │
+└───────────────────────────────────────────────────────────────────────────────┘
+```
+
+### Text hierarchy (L1 / L2 / L3)
+
+Each drone image has a VLM-generated caption (Qwen3-VL) split into 3 levels:
+
+| Level | Name | Content | Typical length |
+|-------|------|---------|----------------|
+| **L1** | Overview | First sentence of P1: land-cover summary with class percentages | 15–30 tokens |
+| **L2** | Full description | Complete P1 (inventory) + P2 (spatial layout with 5+ zones) | 100–200 tokens |
+| **L3** | Fingerprint | P3: unique landmarks and spatial signature for matching | 20–50 tokens |
+
+All three levels are encoded by a **single DGTRS-CLIP ViT-L-14** text encoder
+(248-token context via KPS positional embedding, 768-dim output)
+adapted with **LoRA** (rank=4 on Q/V in all 12 blocks).
+
+### Text fusion (shared MLP for both branches)
+
+```math
+\mathbf{z}_{\text{text}} = \text{MLP}\bigl([\mathbf{z}_1 \;;\; \mathbf{z}_2 \;;\; \mathbf{z}_3]\bigr)
+```
+
+where `[z₁ ; z₂ ; z₃] ∈ ℝ^{B×2304}` is the concatenation of three 768-dim DGTRS-CLIP embeddings, and
+
+```math
+\text{MLP}: \text{Linear}(2304, 512) \to \text{GELU} \to \text{Linear}(512, 512), \quad \mathbf{z}_{\text{text}} \in \mathbb{R}^{B \times 512}
+```
+
+### Gated fusion (separate gates for query and gallery)
+
+```math
+\mathbf{q} = \sigma(\alpha_q) \cdot \mathbf{d}_{\text{img}} + \bigl(1 - \sigma(\alpha_q)\bigr) \cdot \mathbf{d}_{\text{txt}} \qquad \text{(query branch)}
+```
+
+```math
+\mathbf{g} = \sigma(\alpha_g) \cdot \mathbf{s}_{\text{img}} + \bigl(1 - \sigma(\alpha_g)\bigr) \cdot \mathbf{s}_{\text{txt}} \qquad \text{(gallery branch)}
+```
+
+- `α_q, α_g` — separate learnable scalars in logit-space, init `σ(α) ≈ 0.7`
+- `σ` — sigmoid function
+- `d_img, s_img ∈ ℝ^{B×512}` — DINOv3+MONA → projection(1024→512)
+- `d_txt, s_txt ∈ ℝ^{B×512}` — fused text embeddings (TextFusionMLP → 512)
+- For satellite images without captions: `s_txt = None → g = s_img`
+
+### Adaptation methods
+
+| Method | Applied to | Where | Params |
+|--------|-----------|-------|--------|
+| **MONA** (CVPR 2025) | DINOv3 ViT-L/16 (shared) | After MSA and MLP in each of 24 blocks | 6.85M |
+| **LoRA** (rank=4) | DGTRS-CLIP text encoder | Q and V projections in all 12 blocks | 147K |
+| **Projection** | After DINOv3 CLS output | Linear(1024→512) | 525K |
+
+**MONA adapter** (per block):
+
+```math
+\mathbf{x} \leftarrow \mathbf{x} + \text{Up}_{64 \to 1024}\!\Bigl(\text{GELU}\bigl(\text{MonaOp}\bigl(\text{Down}_{1024 \to 64}(\hat{\mathbf{x}})\bigr)\bigr)\Bigr)
+```
+
+where `x̂ = γ · LN(x) + γₓ · x` (scaled LayerNorm, `γ` init `10⁻⁶`, `γₓ` init `1`)
+
+```math
+\text{MonaOp}(\mathbf{x}) = \frac{\text{DWConv}_{3 \times 3}(\mathbf{x}) + \text{DWConv}_{5 \times 5}(\mathbf{x}) + \text{DWConv}_{7 \times 7}(\mathbf{x})}{3} + \mathbf{x}
+```
+
+MONA runs in **bfloat16** with gradient checkpointing to save VRAM.
+Applied only to the **last 12 blocks** (out of 24) — early blocks extract low-level
+features (edges, textures) that are domain-agnostic and don't need spatial adaptation.
+
+**Why bfloat16, not fp16:**
+
+MONA's scaled LayerNorm uses `gamma` initialized at `1e-6` for near-identity output at start.
+fp16 has min normal ~6.1e-5, so `1e-6` falls into the subnormal range where precision collapses,
+causing NaN after a few blocks. bfloat16 has the same exponent range as fp32 (min ~1.2e-38),
+so `1e-6` is safely representable. RTX 4090 supports bf16 natively with comparable throughput.
+
+| Precision | `1e-6` representable | MONA stable | VRAM (bs=48) |
+|-----------|:---:|:---:|:---:|
+| fp32 | yes | yes | 21.4 GB |
+| **bfloat16** | **yes** | **yes** | **21.8 GB** |
+| fp16 | subnormal (lossy) | **NaN** | — |
+
+**Why MONA over LoRA for DINOv3:**
+
+MONA uses multi-scale depthwise convolutions (3×3, 5×5, 7×7) that provide **spatial inductive bias**
+critical for cross-view geo-localization. Drone images (oblique, 100-600m altitude) and satellite
+images (nadir) exhibit a strong **geometric domain gap** — the same building looks spatially different
+from each viewpoint. MONA's multi-scale spatial filters learn scale-invariant features to bridge
+this gap, while LoRA (pure linear low-rank correction) would only handle style/distribution shifts.
+
+| | MONA | LoRA (on DINOv3) |
+|---|---|---|
+| Inductive bias | 2D spatial (knows about pixel neighbors) | None (linear correction) |
+| Best for | Geometric domain gap (aerial↔satellite) | Style/distribution shift |
+| Params | 6.85M (bottleneck=64) | ~0.3M (rank=4) |
+| Compute | Heavier (192 conv ops per forward) | Light |
+| CVGL fit | Strong (multi-scale spatial adaptation) | Weak (no spatial awareness) |
+
+**LoRA** (per DGTRS-CLIP attention layer):
+
+```math
+\mathbf{Q}' = \mathbf{Q} + \frac{\alpha}{r} \cdot \mathbf{x} \mathbf{A}_Q^T \mathbf{B}_Q^T, \qquad \mathbf{V}' = \mathbf{V} + \frac{\alpha}{r} \cdot \mathbf{x} \mathbf{A}_V^T \mathbf{B}_V^T
+```
+
+where `A ∈ ℝ^{r×d}`, `B ∈ ℝ^{d×r}`, `r = 4`. LoRA is appropriate for the text encoder since
+text has no spatial structure — the adaptation needed is purely semantic/distributional.
+
+### Projection head (DINOv3 1024 → 512)
+
+```math
+\mathbf{h} = \text{Linear}_{1024 \to 512}\!\bigl(\text{CLS}_{\text{DINOv3}}\bigr)
+```
+
+Reduces the retrieval space from DINOv3 native 1024-dim to 512-dim. Benefits:
+- Smaller similarity matrix in InfoNCE (`B×B` at 512 vs 1024)
+- TextFusionMLP outputs 512 instead of 1024 (fewer params: 1.5M vs 3.4M)
+- Shared projection for both drone and satellite branches
+
+### Pair formation and negative sampling
+
+The dataset provides only **positive pairs** — each entry maps one drone image to its
+matching satellite crop(s). Negative pairs are **not** stored explicitly; instead, they
+are constructed automatically inside each training batch via the InfoNCE loss:
+
+```
+Batch (B = 8):
+  drone_0 ↔ sat_0  ← positive (from dataset)
+  drone_1 ↔ sat_1  ← positive
+  ...
+  drone_7 ↔ sat_7  ← positive
+
+Similarity matrix B×B:
+       sat_0  sat_1  ...  sat_7
+  q_0 [ pos    neg   ...   neg ]   ← CE target = 0
+  q_1 [ neg    pos   ...   neg ]   ← CE target = 1
+  ...
+  q_7 [ neg    neg   ...   pos ]   ← CE target = 7
+```
+
+- **Positives:** diagonal `sim[i, i]` — correct drone-satellite pair
+- **Negatives:** off-diagonal `sim[i, j≠i]` — other satellite images in the batch (in-batch negatives)
+- At `batch_size=8`: 1 positive + 7 in-batch negatives per query
+- No hard-negative mining — negatives are random within the batch
+- Larger batch → more negatives → harder contrastive task
+
+Each drone image may have multiple satellite candidates (with distance-based weights).
+At training time, one satellite crop is sampled per drone (weighted if semi-positive).
+
+### Loss function
+
+Symmetric InfoNCE with learnable temperature (CLIP-style `logit_scale`):
+
+```math
+\mathcal{L} = w_{q \to g} \cdot \mathcal{L}_{q \to g} + w_{g \to q} \cdot \mathcal{L}_{g \to q}
+```
+
+```math
+\mathcal{L}_{q \to g} = \text{CrossEntropy}\!\left(\frac{\hat{\mathbf{q}} \cdot \hat{\mathbf{g}}^T}{\tau},\; \text{targets}\right), \qquad \mathcal{L}_{g \to q} = \text{CrossEntropy}\!\left(\frac{\hat{\mathbf{g}} \cdot \hat{\mathbf{q}}^T}{\tau},\; \text{targets}\right)
+```
+
+- `τ = 1 / exp(logit_scale)` — learnable scalar, clamped `τ ∈ [0.01, 0.5]`, init `τ₀ = 0.07`
+- `w_{q→g} = 0.6`, `w_{g→q} = 0.4`
+- `targets = [0, 1, 2, ..., B−1]` — positives on diagonal
+- label smoothing `= 0.1`
+
+Loss and adapters run in **fp32** (AMP autocast disabled) to prevent gradient overflow.
+
+### Gradient checkpointing
+
+Gradient checkpointing trades compute for VRAM by recomputing activations during
+backward instead of storing them. Enabled by default (`gradient_checkpointing=True`).
+
+| Component | Without checkpointing | With checkpointing |
+|-----------|:---:|:---:|
+| DINOv3 (24 blocks) | stores all 24 block activations | recomputes on backward |
+| DGTRS-CLIP (12 blocks) | stores all 12 block activations | recomputes on backward |
+| **Max batch_size** (RTX 4090, shared encoder) | **8** | **24** |
+| Speed penalty | — | ~20-30% slower per step |
+
+VRAM tested on RTX 4090 (24 GB) with shared DINOv3 WEB + MONA top-12 bf16 + DGTRS-CLIP + text:
+
+| `batch_size` | Peak VRAM | Status |
+|:---:|:---:|:---:|
+| 32 | 16.1 GB | OK |
+| 40 | 19.1 GB | OK |
+| **48** | **21.8 GB** | **OK (recommended)** |
+| 56 | >24 GB | OOM |
+
+### Gradient accumulation
+
+Gradient accumulation emulates a larger effective batch without extra memory:
+
+```
+effective_batch_size = batch_size × grad_accum_steps
+```
+
+| Setting | `batch_size` | `grad_accum_steps` | Effective batch | In-batch negatives |
+|---------|:---:|:---:|:---:|:---:|
+| Default | 48 | 1 | 48 | 47 |
+| Large effective batch | 48 | 4 | 192 | 47 per micro-batch |
+
+**Note:** gradient accumulation averages gradients across micro-batches, but each
+micro-batch still only sees `batch_size` in-batch negatives. To increase the number
+of negatives per forward pass, increase `batch_size` directly (requires more VRAM).
+
+```bash
+# Example: effective batch of 192 with gradient accumulation
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --grad-accum 4
+```
+
+### Metrics
+
+| Metric | Formula | Direction | Computed on |
+|--------|---------|-----------|:-----------:|
+| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | q→g and g→q | train + val |
+| **AP** (Average Precision) | mean of 1/(rank+1) across all queries (MRR) | q→g and g→q | train + val |
+| **Loss** (InfoNCE) | symmetric cross-entropy on similarity matrix | — | train + val |
+| **Delta R@1** | R@1(with_text) − R@1(baseline) | q→g | val only |
+
+R@1, R@5, R@10, AP, and loss are computed on both **train** (subset matching test size, clean
+transforms) and **val** (full test set) every epoch. Train vs val comparison enables overfitting
+detection: if train R@1/AP rises while val stagnates → overfitting.
+
+**Output CSVs:**
+
+| File | Content | Updated |
+|------|---------|---------|
+| `logs/train.csv` | Epoch-level train loss, temperature, gates, lr | Every epoch |
+| `logs/val.csv` | Val R@K, AP, loss, gates | Every eval epoch |
+| `logs/train_recall.csv` | Train R@K, AP, loss (subset) | Every eval epoch |
+| `logs/train_batches.csv` | Per-batch loss, temperature, gates, lr | Every batch |
+
+**Plots** (auto-generated in `logs/`):
+
+| Plot | Panels |
+|------|--------|
+| `train_metrics.png` | Loss, temperature (τ), gates (σ(α)), learning rate |
+| `val_metrics.png` | R@K q→g (train vs val), R@K g→q (train vs val), AP (train vs val) |
+| `overview.png` | Train+val loss, val R@1, gates + temperature |
+
+### Optimizer & scheduler
+
+```
+Optimizer: AdamW
+  - MONA adapters, projection, TextFusionMLP, gate α_q, gate α_g, logit_scale:
+      lr = 1e-4,  weight_decay = 1e-4
+  - LoRA adapters (DGTRS-CLIP text encoder):
+      lr = 1e-5  (10× lower, --text-lr-factor 0.1)
+
+Scheduler: Linear warmup (2 epochs) + cosine annealing
+  - Per optimizer step (accounts for gradient accumulation)
+  - warmup: lr linearly ramps from 0 to lr_max over warmup_steps
+  - cosine: lr decays from lr_max to 0 over remaining steps
+
+Gradient clipping: max_norm = 1.0
+Gradient accumulation: configurable (default 1, --grad-accum N)
+Mixed precision: AMP fp16 for DINOv3/DGTRS forward, bf16 for MONA, fp32 for loss
+Gradient checkpointing: DINOv3 (24 blocks) + DGTRS-CLIP (12 blocks)
+```
+
+### Augmentations
+
+| Transform | Drone (train) | Satellite (train) | Eval |
+|-----------|:---:|:---:|:---:|
+| RandomResizedCrop(256, scale=0.7–1.0) | ✓ | ✓ | — |
+| Resize(256) + CenterCrop(256) | — | — | ✓ |
+| RandomHorizontalFlip(0.5) | ✓ | ✓ | — |
+| RandomRotation(15°) | ✓ | — | — |
+| ColorJitter(0.3, 0.3, 0.2, 0.1) | ✓ | ✓ | — |
+| RandomGrayscale(0.05) | ✓ | ✓ | — |
+| GaussianBlur(k=3, σ=0.1–2.0) | ✓ | — | — |
+| ImageNet Normalize | ✓ | ✓ | ✓ |
+
+### Model summary
+
+| Component | Params | Trainable | Notes |
+|-----------|--------|-----------|-------|
+| DINOv3 ViT-L/16 WEB (shared) | 303M | frozen | single encoder for drone + satellite |
+| MONA adapters (shared, bf16) | 3.5M | 3.5M | 2 per block × last 12 blocks, bottleneck=64 |
+| Image projection | 525K | 525K | Linear(1024→512) after DINOv3 CLS |
+| DGTRS-CLIP ViT-L-14 (text) | 124M | frozen | backbone weights frozen |
+| LoRA adapters (text) | 147K | 147K | Q+V, rank=4, 12 blocks |
+| TextFusionMLP (shared) | 1.5M | 1.5M | Linear(2304,512) + GELU + Linear(512,512) |
+| GatedFusion α_q + α_g | 2 | 2 | separate gate scalars |
+| logit_scale | 1 | 1 | learnable temperature |
+| **Total (shared)** | **432M** | **5.6M (1.30%)** | retrieval dim = 512 |
+
+> **Asymmetric mode** (`shared_encoder=False`): separate DINOv3 WEB (drone) + DINOv3 SAT
+> (satellite) encoders with independent MONA adapters. Requires ~4-5 GB more VRAM.
+> Use `conf/gtauav_balanced_asym.gin` / `conf/gtauav_baseline_asym.gin` — these set
+> `shared_encoder=False` and `mona_last_n_blocks=24` for the full-capacity setup.
+
+### Eval directions
+
+`_evaluate` computes R@1/5/10 and AP (MRR) for both retrieval directions:
+
+| Key | Direction | Notes |
+|-----|-----------|-------|
+| `r@K_q2g`, `ap_q2g` | drone → satellite (query → gallery) | Denominator: all queries (q2g convention) |
+| `r@K_g2q`, `ap_g2q` | satellite → drone (gallery → query) | Denominator: only sat-tiles with ≥1 positive drone in subsample |
+
+g2q is computed via inverted ground truth: for each sat-tile, collect the drone indices
+that list it as a valid candidate. `n_scored_g2q` is reported in metrics for transparency.
+
+## Experiments
+
+### V3 — GTA-UAV + DINOv3 + DGTRS-CLIP (active)
+
+**Dataset:** GTA-UAV-LR (33K drone + 14K satellite, GTA V synthetic)
+- RGB: `/home/servml/Документы/datasets/GTA-UAV-LR/`
+- Captions: `/home/servml/Документы/datasets/GTA-UAV-LR-captions/` (40K JSON, 3-paragraph VLM)
+- Segmentation: `/home/servml/Документы/datasets/GTA-UAV-LR-aug/` (17 classes)
+- Seg filter: 37,498 passed / 10,905 excluded (>=90% background+water)
+- Split: 80/20 random (26,966 train / 6,742 test → 24,891/6,252 after seg filter)
+
+### V2 — UAV-GeoLoc + GeoRSCLIP (legacy)
+
+Single backbone with template captions.
+
+```
+Query:   drone_img + caption -> GeoRSCLIP -> GatedFusion -> query
+Gallery: sat_img -> GeoRSCLIP -> gallery
+```
+
+**Dataset:** UAV-GeoLoc Terrain split (206K train queries)
+
+## Structure
+
+```
+caption-test/
+├── conf/                         # Gin configs
+│   ├── gtauav_balanced.gin       # GTA-UAV with text, shared encoder, MONA 12/24 (v3)
+│   ├── gtauav_baseline.gin       # GTA-UAV baseline, shared, MONA 12/24, no text (v3)
+│   ├── gtauav_balanced_asym.gin  # GTA-UAV with text, asymmetric WEB+SAT, MONA 24/24
+│   ├── gtauav_baseline_asym.gin  # GTA-UAV baseline, asymmetric, MONA 24/24, no text
+│   ├── gtauav_text_heavy.gin     # GTA-UAV text-heavy gate=0.3 (v3)
+│   ├── gtauav_image_heavy.gin    # GTA-UAV image-heavy gate=0.9 (v3)
+│   ├── balanced.gin              # UAV-GeoLoc with text (v2)
+│   ├── baseline_no_text.gin      # UAV-GeoLoc baseline (v2)
+│   └── text_heavy.gin            # UAV-GeoLoc text-heavy (v2)
+├── nn_models/                    # Pre-trained checkpoints (v3, gitignored)
+│   ├── DINO_WEB/                 # DINOv3 ViT-L/16 LVD-1689M (.pth)
+│   ├── DINO_SAT/                 # DINOv3 ViT-L/16 SAT-493M (.safetensors)
+│   └── LRSCLIP/                  # DGTRS-CLIP ViT-L-14 (.pt)
+├── meta/                         # Generated metadata
+│   ├── train_80.json             # 80% train split (26,966 pairs)
+│   ├── test_20.json              # 20% test split (6,742 pairs)
+│   └── seg_filter.json           # Segmentation filter results
+├── scripts/
+│   ├── make_split.py             # Create 80/20 train/test split
+│   ├── filter_segmentation.py    # Exclude 90%+ background/water images
+│   ├── compare_runs.py           # Delta R@1 comparison report
+│   └── generate_captions.py      # Offline caption generation
+├── src/
+│   ├── datasets/
+│   │   ├── gtauav_dataset.py     # GTA-UAV-LR loader + L1/L2/L3 parsing (v3)
+│   │   └── visloc_with_captions.py  # UAV-GeoLoc loader (v2)
+│   ├── models/
+│   │   ├── dgtrs/                # Official DGTRS-CLIP text encoder (Apache-2.0)
+│   │   │   ├── model.py          # DGTRSTextEncoder, build_model, tokenize
+│   │   │   ├── simple_tokenizer.py  # BPE tokenizer (248 tokens)
+│   │   │   └── bpe_simple_vocab_16e6.txt.gz
+│   │   │   ├── adapters.py          # MONA (bf16) + LoRA adapters
+│   │   ├── asymmetric_encoder.py # DINOv3 + projection + DGTRS + GatedFusion (v3)
+│   │   └── dual_encoder.py       # GeoRSCLIP + GatedFusion (v2)
+│   ├── losses/
+│   │   └── multi_infonce.py      # InfoNCE with learnable temperature
+│   ├── training/
+│   │   ├── train_gtauav.py       # Training loop GTA-UAV (v3)
+│   │   ├── train.py              # Training loop UAV-GeoLoc (v2)
+│   │   ├── trackers.py           # Unified tracker: W&B + TensorBoard
+│   │   ├── grad_monitor.py       # Gradient norm monitoring per group
+│   │   ├── gradcam.py            # Grad-CAM visualization for DINOv3
+│   │   ├── profiling.py          # PyTorch Profiler + torchinfo summary
+│   │   └── plot_metrics.py       # Seaborn/matplotlib metric plots
+│   └── eval/
+│       └── evaluate.py           # R@K metrics, Delta R@1
+└── checkpoints/                  # GeoRSCLIP RS5M_ViT-B-32.pt (v2)
+```
+
+## Prerequisites
+
+```
+torch>=2.0
+safetensors
+coloredlogs
+tqdm
+ftfy
+regex
+gin-config
+Pillow
+numpy
+pandas
+matplotlib
+seaborn
+```
+
+### Optional (for extended diagnostics)
+
+```
+wandb          # Weights & Biases experiment tracking
+torchinfo      # Model summary tables
+tensorboard    # TensorBoard logging (included with torch)
+```
+
+## Workflow (V3 — GTA-UAV)
+
+### 1. Create 80/20 split and filter segmentation
+
+```bash
+python -m scripts.make_split --output-dir meta
+python -m scripts.filter_segmentation --output meta/seg_filter.json
+```
+
+### 2. Train with gin configs (recommended)
+
+```bash
+# With captions (L1/L2/L3, bs=48, 30 epochs, eval every epoch)
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --epochs 30 \
+    --gin-param 'TrainConfigGTAUAV.eval_every=1'
+
+# Baseline (no text)
+python -m src.training.train_gtauav --config conf/gtauav_baseline.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --epochs 30
+
+# Text-heavy (gate=0.3, 70% text weight)
+python -m src.training.train_gtauav --config conf/gtauav_text_heavy.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --epochs 30
+
+# Image-heavy (gate=0.9, 10% text weight)
+python -m src.training.train_gtauav --config conf/gtauav_image_heavy.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --epochs 30
+
+# Asymmetric variants: separate WEB (drone) + SAT (satellite) encoders, MONA in all 24 blocks
+# Higher capacity (~733M total / ~17.6M trainable), larger VRAM footprint.
+python -m src.training.train_gtauav --config conf/gtauav_balanced_asym.gin \
+    --filter-meta meta/seg_filter.json
+python -m src.training.train_gtauav --config conf/gtauav_baseline_asym.gin \
+    --filter-meta meta/seg_filter.json
+```
+
+### 3. Train without gin (CLI-only)
+
+```bash
+python -m src.training.train_gtauav --baseline --filter-meta meta/seg_filter.json --batch-size 48
+python -m src.training.train_gtauav --filter-meta meta/seg_filter.json --batch-size 48
+```
+
+### 4. Enable diagnostics
+
+```bash
+# W&B + Grad-CAM + PyTorch Profiler
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json --batch-size 48 --wandb --gradcam --profile
+
+# Gin parameter overrides from CLI
+python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
+    --filter-meta meta/seg_filter.json \
+    --gin-param 'TrainConfigGTAUAV.eval_every=1' 'TrainConfigGTAUAV.epochs=30'
+```
+
+CLI flags (`--wandb`, `--gradcam`, `--profile`, `--epochs`, `--batch-size`, etc.) take priority over gin config.
+
+### 5. Resume from checkpoint
+
+```bash
+python -m src.training.train_gtauav --resume out/gtauav/with_text/ckpt_epoch004.pt \
+    --filter-meta meta/seg_filter.json
+```
+
+### 6. Compare and get verdict
+
+```bash
+python -m scripts.compare_runs \
+    --baseline_report out/gtauav/baseline/eval_report.json \
+    --full_report out/gtauav/with_text/eval_report.json \
+    --output out/gtauav/comparison.md
+```
+
+### 7. View TensorBoard
+
+```bash
+tensorboard --logdir out/gtauav/with_text/tb_logs
+```
+
+## Diagnostics & Visualization
+
+| Tool | Flag | Output | Description |
+|------|------|--------|-------------|
+| **TensorBoard** | `--use-tb` (default on) | `{out}/tb_logs/` | Scalars, histograms, images |
+| **W&B** | `--wandb` | cloud | Full experiment tracking, Grad-CAM images |
+| **Grad-CAM** | `--gradcam` | `{out}/gradcam/` | DINOv3 attention heatmaps (drone + satellite) |
+| **PyTorch Profiler** | `--profile` | `{out}/profiler/` | Chrome trace, CUDA timeline, memory |
+| **torchinfo** | auto | `{out}/model_summary.txt` | Layer-by-layer parameter table |
+| **Gradient norms** | `--log-grad-norms` (default on) | TB/W&B | Per-group: MONA, LoRA, MLP, gates, tau |
+| **CSV (per-batch)** | auto | `{out}/logs/train_batches.csv` | Loss, tau, gates, lr for every batch |
+| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch loss averages + seaborn plots |
+| **CSV (recall)** | auto | `{out}/logs/train_recall.csv` | Train R@K, AP, loss (subset, clean transforms) |
+| **Plots** | auto | `{out}/logs/*.png` | train/val loss, R@K, AP, gates, temperature |
+
+## Decision rule
+
+| Delta R@1 (drone→satellite) | Verdict |
+|---|---|
+| >= +3% | **PASS** — captions informative, proceed to NADEZHDA teacher |
+| +1% to +3% | MARGINAL — add VLM refinement, re-run |
+| 0 to +1% | WEAK — redesign caption pipeline |
+| < 0 | HARMFUL — critical bug |
+
+## Code style
+
+- `from __future__ import annotations` everywhere
+- Type hints on all signatures
+- Google-style docstrings
+- English-only comments
--- a/vendor_reference/caption_test/conf/gtauav_balanced_stripnet.gin
+++ b/vendor_reference/caption_test/conf/gtauav_balanced_stripnet.gin
@@ -0,0 +1,27 @@
+# GTA-UAV Balanced (StripNet backbone): StripNet-small + Conv-MONA in last 2 stages.
+# Replaces DINOv3 ViT-L/16 with strip-shaped DWConv hierarchical CNN (~28M params,
+# 10× smaller than DINOv3). Output 512-dim → projected to 1024 to match retrieval space.
+#
+# Trainable:
+#   - Projection (Linear 512→1024): ~525K
+#   - Conv-MONA in stages 3 & 4 (2 adapters per Block × 6 blocks total): ~2-3M
+#   - LoRA on DGTRS-CLIP: 147K
+#   - TextFusionMLP (shared): ~3.4M
+#   - GatedFusion gates + tau: 3 scalars
+#
+# StripNet pretrained on ImageNet-1K (head dropped); state-dict naming follows
+# upstream Strip-R-CNN repo (`conv_spatial1/2`).
+
+include 'conf/gtauav_balanced.gin'
+
+# ---- Backbone ----
+TrainConfigGTAUAV.backbone = "stripnet"
+TrainConfigGTAUAV.stripnet_path = "nn_models/STRIPNET/stripnet_s.pth"
+TrainConfigGTAUAV.stripnet_mona_last_n_stages = 2  # Conv-MONA in stages 3 & 4 (deepest)
+
+# ---- Model overrides ----
+TrainConfigGTAUAV.shared_encoder = True   # StripNet always shared (one CNN for both branches)
+TrainConfigGTAUAV.mona_bottleneck = 64    # Conv-MONA bottleneck channels
+
+# ---- Output ----
+TrainConfigGTAUAV.output_dir = "out/gtauav/with_text_stripnet"
--- a/vendor_reference/caption_test/conf/gtauav_baseline_stripnet.gin
+++ b/vendor_reference/caption_test/conf/gtauav_baseline_stripnet.gin
@@ -0,0 +1,8 @@
+# GTA-UAV Baseline (StripNet backbone): no text fusion. Reference R@1 for
+# computing Δ R@1 against gtauav_balanced_stripnet.gin.
+
+include 'conf/gtauav_balanced_stripnet.gin'
+
+TrainConfigGTAUAV.baseline_mode = True
+TrainConfigGTAUAV.output_dir = "out/gtauav/baseline_stripnet"
+TrainConfigGTAUAV.use_gradcam = False
--- a/vendor_reference/caption_test/scripts/filter_segmentation.py
+++ b/vendor_reference/caption_test/scripts/filter_segmentation.py
@@ -0,0 +1,140 @@
+from __future__ import annotations
+
+"""Filter GTA-UAV-LR images by segmentation class coverage.
+
+Reads palette-mode PNG segmentation masks, computes per-class pixel ratios,
+and outputs a JSON meta file listing images that pass the filter
+(i.e. background + water < threshold).
+
+Classes (from manifest.json):
+    0: background, 1: building, 2: road, 3: vegetation, 4: water, ...
+
+Usage:
+    python -m scripts.filter_segmentation [--threshold 0.9] [--output meta/seg_filter.json]
+"""
+
+import argparse
+import json
+import logging
+from pathlib import Path
+
+import coloredlogs
+import numpy as np
+from PIL import Image
+from tqdm import tqdm
+
+LOGGER = logging.getLogger("caption_test.filter_seg")
+
+SEGM_ROOT = Path("/home/servml/Документы/datasets/GTA-UAV-LR-aug/segm")
+EXCLUDE_CLASSES = {0, 4}  # background, water
+DEFAULT_THRESHOLD = 0.90
+
+
+def compute_class_ratios(mask_path: Path) -> dict[int, float]:
+    """Load a palette-mode PNG mask and return per-class pixel ratios."""
+    with Image.open(mask_path) as img:
+        arr = np.array(img)
+    total = arr.size
+    unique, counts = np.unique(arr, return_counts=True)
+    return {int(cls): float(cnt / total) for cls, cnt in zip(unique, counts)}
+
+
+def scan_masks(
+    segm_root: Path,
+    exclude_classes: set[int],
+    threshold: float,
+) -> dict[str, dict]:
+    """Scan all segmentation masks and classify as pass/fail.
+
+    Returns:
+        Dict keyed by relative image name (e.g. "drone/images/100_0001_0000000000.png")
+        with values: {"ratios": {...}, "excluded_ratio": float, "pass": bool}
+    """
+    results: dict[str, dict] = {}
+    subdirs = sorted(p for p in segm_root.iterdir() if p.is_dir())
+
+    for subdir in subdirs:
+        mask_files = sorted(subdir.rglob("*.png"))
+        LOGGER.info("🔍 Scanning %s: %d masks", subdir.name, len(mask_files))
+
+        for mask_path in tqdm(mask_files, desc=f"  📂 {subdir.name}", unit="img", leave=False):
+            rel = mask_path.relative_to(segm_root)
+            ratios = compute_class_ratios(mask_path)
+            excluded_ratio = sum(ratios.get(c, 0.0) for c in exclude_classes)
+            results[str(rel)] = {
+                "ratios": ratios,
+                "excluded_ratio": round(excluded_ratio, 6),
+                "pass": excluded_ratio < threshold,
+            }
+
+    return results
+
+
+def build_meta(
+    results: dict[str, dict],
+) -> dict[str, list[str]]:
+    """Split results into passed and excluded lists."""
+    passed = sorted(k for k, v in results.items() if v["pass"])
+    excluded = sorted(k for k, v in results.items() if not v["pass"])
+    return {"passed": passed, "excluded": excluded}
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Filter GTA-UAV-LR images by segmentation coverage.",
+    )
+    parser.add_argument(
+        "--segm-root", type=str, default=str(SEGM_ROOT),
+        help="Root of segmentation masks.",
+    )
+    parser.add_argument(
+        "--threshold", type=float, default=DEFAULT_THRESHOLD,
+        help="Exclude images where background+water >= threshold.",
+    )
+    parser.add_argument(
+        "--output", type=str, default="meta/seg_filter.json",
+        help="Output JSON meta file path.",
+    )
+    args = parser.parse_args()
+
+    coloredlogs.install(
+        level="INFO",
+        logger=LOGGER,
+        fmt="%(asctime)s %(name)s %(levelname)s %(message)s",
+    )
+
+    LOGGER.info("🚀 Starting segmentation filter (threshold=%.2f)", args.threshold)
+
+    segm_root = Path(args.segm_root)
+    results = scan_masks(segm_root, EXCLUDE_CLASSES, args.threshold)
+    meta = build_meta(results)
+
+    n_total = len(results)
+    n_pass = len(meta["passed"])
+    n_excl = len(meta["excluded"])
+    LOGGER.info(
+        "📊 total=%d  ✅ passed=%d (%.1f%%)  ❌ excluded=%d (%.1f%%)",
+        n_total, n_pass, 100 * n_pass / max(n_total, 1),
+        n_excl, 100 * n_excl / max(n_total, 1),
+    )
+
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    out = {
+        "threshold": args.threshold,
+        "exclude_classes": sorted(EXCLUDE_CLASSES),
+        "total_images": n_total,
+        "passed_count": n_pass,
+        "excluded_count": n_excl,
+        "passed": meta["passed"],
+        "excluded": meta["excluded"],
+    }
+    with output_path.open("w", encoding="utf-8") as f:
+        json.dump(out, f, indent=2)
+
+    LOGGER.info("💾 Meta file saved to %s", output_path)
+
+
+if __name__ == "__main__":
+    main()
--- a/vendor_reference/caption_test/scripts/make_split.py
+++ b/vendor_reference/caption_test/scripts/make_split.py
@@ -0,0 +1,87 @@
+from __future__ import annotations
+
+"""Create 80/20 train/test split from GTA-UAV-LR pair JSONs.
+
+Merges cross-area train+test (33,708 pairs), shuffles deterministically,
+and saves new 80/20 split JSONs.
+
+Usage:
+    python -m scripts.make_split [--ratio 0.8] [--seed 42]
+"""
+
+import argparse
+import json
+import logging
+import random
+from pathlib import Path
+
+import coloredlogs
+
+LOGGER = logging.getLogger("caption_test.make_split")
+
+_RGB_ROOT = Path("/home/servml/Документы/datasets/GTA-UAV-LR")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Create 80/20 split for GTA-UAV-LR.")
+    parser.add_argument("--ratio", type=float, default=0.8, help="Train ratio (default 0.8).")
+    parser.add_argument("--seed", type=int, default=42, help="Random seed.")
+    parser.add_argument(
+        "--output-dir", type=str, default="meta",
+        help="Output directory for split JSONs.",
+    )
+    args = parser.parse_args()
+
+    coloredlogs.install(
+        level="INFO", logger=LOGGER,
+        fmt="%(asctime)s %(name)s %(levelname)s %(message)s",
+    )
+
+    # Load both original splits.
+    train_path = _RGB_ROOT / "cross-area-drone2sate-train.json"
+    test_path = _RGB_ROOT / "cross-area-drone2sate-test.json"
+
+    LOGGER.info("📂 Loading %s", train_path.name)
+    with open(train_path) as f:
+        part1 = json.load(f)
+    LOGGER.info("📂 Loading %s", test_path.name)
+    with open(test_path) as f:
+        part2 = json.load(f)
+
+    all_pairs = part1 + part2
+    LOGGER.info("📊 Total pairs: %d", len(all_pairs))
+
+    # Shuffle deterministically.
+    rng = random.Random(args.seed)
+    rng.shuffle(all_pairs)
+
+    # Split.
+    n_train = int(len(all_pairs) * args.ratio)
+    train_pairs = all_pairs[:n_train]
+    test_pairs = all_pairs[n_train:]
+
+    LOGGER.info(
+        "✂️  Split %.0f/%.0f: train=%d (%.1f%%)  test=%d (%.1f%%)",
+        args.ratio * 100, (1 - args.ratio) * 100,
+        len(train_pairs), 100 * len(train_pairs) / len(all_pairs),
+        len(test_pairs), 100 * len(test_pairs) / len(all_pairs),
+    )
+
+    # Save.
+    out_dir = Path(args.output_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    train_out = out_dir / "train_80.json"
+    test_out = out_dir / "test_20.json"
+
+    with train_out.open("w", encoding="utf-8") as f:
+        json.dump(train_pairs, f)
+    with test_out.open("w", encoding="utf-8") as f:
+        json.dump(test_pairs, f)
+
+    LOGGER.info("💾 Saved: %s (%d pairs)", train_out, len(train_pairs))
+    LOGGER.info("💾 Saved: %s (%d pairs)", test_out, len(test_pairs))
+
+
+if __name__ == "__main__":
+    main()
--- a/vendor_reference/caption_test/src/eval/evaluate.py
+++ b/vendor_reference/caption_test/src/eval/evaluate.py
@@ -0,0 +1,153 @@
+from __future__ import annotations
+
+"""Evaluation for caption quality test.
+
+Recall@K for query(drone+text) -> gallery(satellite).
+delta_r_at_1 compares caption-aware vs baseline runs.
+"""
+
+import json
+import logging
+from pathlib import Path
+
+import gin
+import torch
+from torch.utils.data import DataLoader
+
+from src.models.dual_encoder import DualEncoderCaptionTest
+
+LOGGER = logging.getLogger("caption_test.eval")
+
+
+def _recall_at_k(
+    similarity: torch.Tensor,
+    k_values: tuple[int, ...] = (1, 5, 10),
+) -> dict[int, float]:
+    """Recall@K assuming positives on the diagonal."""
+    n_query = similarity.size(0)
+    targets = torch.arange(n_query, device=similarity.device)
+    sorted_idx = similarity.argsort(dim=1, descending=True)
+    result: dict[int, float] = {}
+    for k in k_values:
+        top_k = sorted_idx[:, :k]
+        hit = (top_k == targets.unsqueeze(1)).any(dim=1).float()
+        result[k] = float(hit.mean().item())
+    return result
+
+
+@torch.no_grad()
+def _encode_dataset(
+    model: DualEncoderCaptionTest,
+    loader: DataLoader,
+    device: str,
+) -> dict[str, torch.Tensor]:
+    """Encode all samples into query and gallery embeddings."""
+    model.eval()
+    all_query: list[torch.Tensor] = []
+    all_gallery: list[torch.Tensor] = []
+
+    for batch in loader:
+        drone_img = batch["drone_img"].to(device, non_blocking=True)
+        sat_img = batch["sat_img"].to(device, non_blocking=True)
+        caption_drone = batch["caption_drone"]
+
+        embeddings = model(
+            drone_img=drone_img,
+            sat_img=sat_img,
+            caption_drone=caption_drone,
+        )
+        all_query.append(embeddings["query"].cpu())
+        all_gallery.append(embeddings["gallery"].cpu())
+
+    return {
+        "query": torch.cat(all_query, dim=0),
+        "gallery": torch.cat(all_gallery, dim=0),
+    }
+
+
+def evaluate_retrieval(
+    model: DualEncoderCaptionTest,
+    loader: DataLoader,
+    device: str,
+    k_values: tuple[int, ...] = (1, 5, 10),
+) -> dict[str, float]:
+    """Compute R@K for query->gallery and gallery->query.
+
+    Returns:
+        Flat dict: r@1_query_to_gallery, r@5_query_to_gallery, etc.
+    """
+    feats = _encode_dataset(model=model, loader=loader, device=device)
+
+    metrics: dict[str, float] = {}
+
+    sim_q2g = feats["query"] @ feats["gallery"].t()
+
+    for k, val in _recall_at_k(sim_q2g, k_values).items():
+        metrics[f"r@{k}_query_to_gallery"] = val
+    for k, val in _recall_at_k(sim_q2g.t(), k_values).items():
+        metrics[f"r@{k}_gallery_to_query"] = val
+
+    # Gate value for diagnostics.
+    metrics["gate"] = model.fusion.gate_value
+
+    return metrics
+
+
+def delta_r_at_1(
+    full_metrics: dict[str, float],
+    baseline_metrics: dict[str, float],
+) -> float:
+    """R@1 gain from adding captions: full - baseline."""
+    key = "r@1_query_to_gallery"
+    return full_metrics[key] - baseline_metrics[key]
+
+
+@gin.configurable
+def run_evaluation_from_checkpoint(
+    checkpoint_path: str,
+    test_query_file: str,
+    data_root: str,
+    output_path: str = "eval_report.json",
+    batch_size: int = 128,
+    num_workers: int = 4,
+    device: str = "cuda",
+) -> dict[str, float]:
+    """Standalone evaluation from checkpoint."""
+    from src.datasets.visloc_with_captions import (
+        GeoLocCaptionDataset,
+        collate_caption_batch,
+    )
+
+    model = DualEncoderCaptionTest().to(device)
+    ckpt = torch.load(checkpoint_path, map_location=device, weights_only=False)
+    model.load_state_dict(ckpt["model_state"])
+    model.eval()
+
+    test_ds = GeoLocCaptionDataset(
+        query_file=test_query_file,
+        data_root=data_root,
+        image_transform=model.preprocess,
+    )
+    test_loader = DataLoader(
+        test_ds,
+        batch_size=batch_size,
+        shuffle=False,
+        num_workers=num_workers,
+        collate_fn=collate_caption_batch,
+        pin_memory=True,
+    )
+
+    metrics = evaluate_retrieval(model=model, loader=test_loader, device=device)
+
+    report = {
+        "checkpoint": checkpoint_path,
+        "test_query_file": test_query_file,
+        "metrics": metrics,
+    }
+    out = Path(output_path)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    with out.open("w", encoding="utf-8") as f:
+        json.dump(report, f, indent=2)
+
+    LOGGER.info("evaluation report saved to %s", out)
+    return metrics
--- a/vendor_reference/caption_test/src/losses/hard_negatives.py
+++ b/vendor_reference/caption_test/src/losses/hard_negatives.py
@@ -0,0 +1,70 @@
+from __future__ import annotations
+
+"""Hard negative memory bank for contrastive learning.
+
+MoCo-style FIFO queue of recent gallery embeddings. Each batch gets
+B in-batch negatives + Q queue negatives, significantly increasing
+the effective number of negatives without extra VRAM for forward pass.
+
+Usage:
+    bank = NegativeMemoryBank(size=4096, dim=1024)
+    # In training loop:
+    sim = bank.compute_similarity(query, gallery)  # [B, B + Q]
+    bank.enqueue(gallery.detach())
+"""
+
+import torch
+import torch.nn as nn
+
+
+class NegativeMemoryBank(nn.Module):
+    """FIFO queue of detached gallery embeddings for hard negatives.
+
+    Args:
+        size: Queue capacity (number of stored embeddings).
+        dim: Embedding dimension.
+    """
+
+    def __init__(self, size: int = 4096, dim: int = 1024) -> None:
+        super().__init__()
+        self.size = size
+        self.dim = dim
+        # Queue stored as buffer (not a parameter, moves with .to(device)).
+        self.register_buffer("queue", torch.randn(size, dim))
+        self.queue = nn.functional.normalize(self.queue, dim=-1)
+        self.register_buffer("ptr", torch.zeros(1, dtype=torch.long))
+        self.register_buffer("full", torch.zeros(1, dtype=torch.bool))
+
+    @torch.no_grad()
+    def enqueue(self, embeddings: torch.Tensor) -> None:
+        """Add embeddings to the queue (FIFO). Oldest are overwritten."""
+        batch_size = embeddings.shape[0]
+        ptr = int(self.ptr.item())
+
+        if ptr + batch_size <= self.size:
+            self.queue[ptr:ptr + batch_size] = embeddings.detach()
+        else:
+            # Wrap around.
+            overflow = (ptr + batch_size) - self.size
+            self.queue[ptr:] = embeddings[:batch_size - overflow].detach()
+            self.queue[:overflow] = embeddings[batch_size - overflow:].detach()
+
+        new_ptr = (ptr + batch_size) % self.size
+        self.ptr[0] = new_ptr
+        if not self.full.item() and (new_ptr < ptr or new_ptr == 0):
+            self.full[0] = True
+
+    def get_queue(self) -> torch.Tensor:
+        """Return valid queue entries [Q, dim]."""
+        if self.full.item():
+            return self.queue
+        ptr = int(self.ptr.item())
+        if ptr == 0:
+            return self.queue[:0]  # empty
+        return self.queue[:ptr]
+
+    @property
+    def current_size(self) -> int:
+        if self.full.item():
+            return self.size
+        return int(self.ptr.item())
--- a/vendor_reference/caption_test/src/losses/multi_infonce.py
+++ b/vendor_reference/caption_test/src/losses/multi_infonce.py
@@ -0,0 +1,203 @@
+from __future__ import annotations
+
+"""InfoNCE loss for cross-view geo-localization with optional text fusion.
+
+Single symmetric InfoNCE between query (drone+text fused) and gallery (satellite).
+Asymmetric weighting: query->gallery weighted higher (real use-case direction).
+
+Supports both learnable temperature (CLIP-style logit_scale) and fixed/scheduled.
+"""
+
+import math
+
+import gin
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+def _symmetric_info_nce(
+    emb_a: torch.Tensor,
+    emb_b: torch.Tensor,
+    temperature: float | torch.Tensor,
+    label_smoothing: float,
+    weight_a2b: float = 0.5,
+    weight_b2a: float = 0.5,
+    queue_negatives: torch.Tensor | None = None,
+    hard_mining_k: int = 0,
+) -> torch.Tensor:
+    """Weighted symmetric InfoNCE with optional hard negative queue.
+
+    Args:
+        emb_a: Query embeddings [B, D].
+        emb_b: Gallery embeddings [B, D]. Positives on diagonal.
+        queue_negatives: Extra gallery negatives [Q, D] from memory bank.
+        hard_mining_k: If > 0 and queue is non-empty, use only the top-K
+            hardest (highest-similarity) queue entries per query instead
+            of the full queue. Per-query selection — each row gets its
+            own K negatives gathered via `topk`.
+    """
+    batch_size = emb_a.size(0)
+    emb_a_f = emb_a.float()
+    emb_b_f = emb_b.float()
+
+    if queue_negatives is not None and queue_negatives.shape[0] > 0:
+        queue_f = queue_negatives.float()
+        sim_inbatch = emb_a_f @ emb_b_f.t() / temperature  # [B, B]
+        sim_queue = emb_a_f @ queue_f.t() / temperature    # [B, Q]
+
+        if hard_mining_k > 0 and hard_mining_k < queue_f.shape[0]:
+            # Per-row top-K — each query gets its own hardest negatives.
+            sim_queue, _ = sim_queue.topk(k=hard_mining_k, dim=1)  # [B, K]
+
+        # a→b: [B, B + (Q or K)]. Positive at column `i` for row `i`.
+        logits_a2b = torch.cat([sim_inbatch, sim_queue], dim=1)
+        targets_a = torch.arange(batch_size, device=emb_a.device)
+        loss_a2b = F.cross_entropy(logits_a2b, targets_a, label_smoothing=label_smoothing)
+
+        # b→a: gallery sees B in-batch queries (queue is gallery-side, irrelevant here).
+        logits_b2a = sim_inbatch.t()  # [B, B]
+        targets_b = torch.arange(batch_size, device=emb_a.device)
+        loss_b2a = F.cross_entropy(logits_b2a, targets_b, label_smoothing=label_smoothing)
+    else:
+        logits = emb_a_f @ emb_b_f.t() / temperature
+        targets = torch.arange(batch_size, device=emb_a.device)
+        loss_a2b = F.cross_entropy(logits, targets, label_smoothing=label_smoothing)
+        loss_b2a = F.cross_entropy(logits.t(), targets, label_smoothing=label_smoothing)
+
+    return weight_a2b * loss_a2b + weight_b2a * loss_b2a
+
+
+def cosine_temperature(
+    epoch: int,
+    total_epochs: int,
+    tau_init: float = 0.1,
+    tau_final: float = 0.01,
+) -> float:
+    """Cosine-decay schedule for InfoNCE temperature."""
+    total_epochs = max(total_epochs, 1)
+    progress = min(max(epoch / total_epochs, 0.0), 1.0)
+    cosine = 0.5 * (1.0 + math.cos(math.pi * progress))
+    return tau_final + (tau_init - tau_final) * cosine
+
+
+@gin.configurable
+class InfoNCELoss(nn.Module):
+    """Symmetric InfoNCE with learnable or scheduled temperature.
+
+    Args:
+        temperature_init: Initial temperature value.
+        temperature_final: Final temperature (only used if learnable=False).
+        label_smoothing: Cross-entropy label smoothing.
+        weight_q2g: Weight for query->gallery direction.
+        weight_g2q: Weight for gallery->query direction.
+        learnable_temperature: If True, temperature is a learnable parameter
+            (CLIP-style logit_scale). If False, uses cosine schedule.
+        tau_min: Minimum clamp for learnable temperature.
+        tau_max: Maximum clamp for learnable temperature.
+        hard_mining_k: If > 0, mine top-K hardest negatives per query from
+            the memory bank queue instead of using the full queue. 0 disables
+            mining (queue used whole). Typical values: 256-1024 for queue=4096.
+    """
+
+    def __init__(
+        self,
+        temperature_init: float = 0.07,
+        temperature_final: float = 0.01,
+        label_smoothing: float = 0.1,
+        weight_q2g: float = 0.6,
+        weight_g2q: float = 0.4,
+        learnable_temperature: bool = True,
+        tau_min: float = 0.01,
+        tau_max: float = 0.1,
+        hard_mining_k: int = 0,
+    ) -> None:
+        super().__init__()
+        self.temperature_init = temperature_init
+        self.temperature_final = temperature_final
+        self.label_smoothing = label_smoothing
+        self.weight_q2g = weight_q2g
+        self.weight_g2q = weight_g2q
+        self.learnable_temperature = learnable_temperature
+        self.tau_min = tau_min
+        self.tau_max = tau_max
+        self.hard_mining_k = hard_mining_k
+
+        if learnable_temperature:
+            # Store as log(1/tau) like CLIP's logit_scale.
+            init_logit_scale = math.log(1.0 / temperature_init)
+            self.logit_scale = nn.Parameter(torch.tensor(init_logit_scale))
+        else:
+            self.logit_scale = None
+
+    @property
+    def current_temperature(self) -> float:
+        """Current temperature value (for logging)."""
+        if self.logit_scale is not None:
+            tau = 1.0 / self.logit_scale.exp().clamp(
+                min=1.0 / self.tau_max, max=1.0 / self.tau_min,
+            ).item()
+            return tau
+        return self.temperature_init
+
+    def forward(
+        self,
+        embeddings: dict[str, torch.Tensor],
+        epoch: int,
+        total_epochs: int,
+        queue_negatives: torch.Tensor | None = None,
+    ) -> dict[str, torch.Tensor]:
+        """Compute InfoNCE loss with optional hard negative queue.
+
+        Args:
+            embeddings: Dict with 'query' and 'gallery' [B, D] L2-normalized.
+            epoch: Current epoch (0-indexed).
+            total_epochs: Total epochs for temperature schedule.
+            queue_negatives: Extra gallery negatives [Q, D] from memory bank.
+
+        Returns:
+            Dict with 'total', 'temperature', 'gate_q', 'gate_g'.
+        """
+        if self.learnable_temperature:
+            # Clamp logit_scale in logit space first to prevent exp() overflow in fp16.
+            # tau_min=0.01 -> max logit_scale=ln(1/0.01)=4.6
+            # tau_max=0.1  -> min logit_scale=ln(1/0.1)=2.30
+            clamped = self.logit_scale.float().clamp(
+                min=math.log(1.0 / self.tau_max),
+                max=math.log(1.0 / self.tau_min),
+            )
+            logit_scale = clamped.exp()
+            tau = 1.0 / logit_scale
+        else:
+            tau = cosine_temperature(
+                epoch=epoch,
+                total_epochs=total_epochs,
+                tau_init=self.temperature_init,
+                tau_final=self.temperature_final,
+            )
+
+        loss = _symmetric_info_nce(
+            emb_a=embeddings["query"],
+            emb_b=embeddings["gallery"],
+            temperature=tau,
+            label_smoothing=self.label_smoothing,
+            weight_a2b=self.weight_q2g,
+            weight_b2a=self.weight_g2q,
+            queue_negatives=queue_negatives,
+            hard_mining_k=self.hard_mining_k,
+        )
+
+        gate_q = embeddings.get("gate_q", embeddings.get("gate", 1.0))
+        gate_g = embeddings.get("gate_g", 1.0)
+
+        if isinstance(tau, float):
+            tau_out = torch.tensor(tau, device=loss.device)
+        else:
+            tau_out = tau.detach().clone()
+
+        return {
+            "total": loss,
+            "temperature": tau_out,
+            "gate_q": torch.tensor(gate_q, device=loss.device),
+            "gate_g": torch.tensor(gate_g, device=loss.device),
+        }
--- a/vendor_reference/caption_test/src/losses/weighted_infonce.py
+++ b/vendor_reference/caption_test/src/losses/weighted_infonce.py
@@ -0,0 +1,148 @@
+from __future__ import annotations
+
+"""Weighted InfoNCE loss for GTA-UAV cross-view geo-localization.
+
+Adapted from Game4Loc (https://github.com/Yux1angJi/GTA-UAV).
+Uses per-sample label smoothing based on positive_weights (IoU/distance)
+to handle partial overlap between drone and satellite crops.
+
+Standard InfoNCE assumes strict 1-to-1 pairs and treats all non-diagonal
+entries as negatives. In GTA-UAV, multiple satellite crops can validly
+match one drone image (partial IoU overlap), causing false negatives.
+WeightedInfoNCE softens this with adaptive label smoothing per sample.
+"""
+
+import math
+
+import gin
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+@gin.configurable
+class WeightedInfoNCELoss(nn.Module):
+    """Weighted InfoNCE with adaptive per-sample label smoothing.
+
+    For each sample i, eps_i = 1 - (1 - base_smoothing) / (1 + exp(-k * w_i))
+    where w_i is the positive weight (e.g. IoU with matched satellite crop).
+    Higher weight → lower eps → sharper target (strong positive).
+    Lower weight → higher eps → softer target (weak/semi-positive).
+
+    Args:
+        temperature_init: Initial temperature (or learnable logit_scale).
+        learnable_temperature: If True, temperature is learnable (CLIP-style).
+        label_smoothing: Base label smoothing (used when no weights provided).
+        k: Sigmoid steepness for weight → eps mapping.
+        tau_min: Min clamp for learnable temperature.
+        tau_max: Max clamp for learnable temperature.
+    """
+
+    def __init__(
+        self,
+        temperature_init: float = 0.07,
+        learnable_temperature: bool = True,
+        label_smoothing: float = 0.1,
+        k: float = 5.0,
+        tau_min: float = 0.01,
+        tau_max: float = 0.1,
+    ) -> None:
+        super().__init__()
+        self.label_smoothing = label_smoothing
+        self.k = k
+        self.tau_min = tau_min
+        self.tau_max = tau_max
+        self.learnable_temperature = learnable_temperature
+
+        if learnable_temperature:
+            self.logit_scale = nn.Parameter(
+                torch.tensor(math.log(1.0 / temperature_init))
+            )
+        else:
+            self.logit_scale = None
+            self.temperature = temperature_init
+
+    @property
+    def current_temperature(self) -> float:
+        if self.logit_scale is not None:
+            tau = 1.0 / self.logit_scale.exp().clamp(
+                min=1.0 / self.tau_max, max=1.0 / self.tau_min,
+            ).item()
+            return tau
+        return self.temperature
+
+    def _compute_eps(self, positive_weights: torch.Tensor | None, n: int) -> torch.Tensor | list[float]:
+        """Compute per-sample label smoothing from positive weights."""
+        if positive_weights is not None:
+            # Higher weight → lower eps (sharper, stronger positive).
+            return 1.0 - (1.0 - self.label_smoothing) / (1.0 + torch.exp(-self.k * positive_weights))
+        return [self.label_smoothing] * n
+
+    def _weighted_loss(
+        self,
+        sim_matrix: torch.Tensor,
+        eps_all: torch.Tensor | list[float],
+    ) -> torch.Tensor:
+        """Weighted InfoNCE: per-sample interpolation between hard and uniform targets.
+
+        For each row i:
+          L_i = (1-eps_i) * [-sim[i,i] + logsumexp(sim[i,:])]
+              + eps_i * [-mean(sim[i,:]) + logsumexp(sim[i,:])]
+        """
+        n = sim_matrix.shape[0]
+        total_loss = torch.tensor(0.0, device=sim_matrix.device)
+        for i in range(n):
+            eps = eps_all[i] if isinstance(eps_all, list) else eps_all[i]
+            logsumexp = torch.logsumexp(sim_matrix[i, :], dim=0)
+            total_loss += (1 - eps) * (-sim_matrix[i, i] + logsumexp)
+            total_loss += eps * (-sim_matrix[i, :].mean() + logsumexp)
+        return total_loss / n
+
+    def forward(
+        self,
+        embeddings: dict[str, torch.Tensor],
+        epoch: int = 0,
+        total_epochs: int = 1,
+        positive_weights: torch.Tensor | None = None,
+        queue_negatives: torch.Tensor | None = None,
+    ) -> dict[str, torch.Tensor]:
+        """Compute weighted InfoNCE loss.
+
+        Args:
+            embeddings: Dict with 'query' [B,D], 'gallery' [B,D], 'gate_q', 'gate_g'.
+            positive_weights: Per-sample weight [B] (e.g. IoU with matched sat crop).
+            queue_negatives: Extra negatives [Q,D] from memory bank (not used with weighted loss).
+        """
+        query = embeddings["query"].float()
+        gallery = embeddings["gallery"].float()
+
+        # Temperature.
+        if self.learnable_temperature:
+            clamped = self.logit_scale.float().clamp(
+                min=math.log(1.0 / self.tau_max),
+                max=math.log(1.0 / self.tau_min),
+            )
+            logit_scale = clamped.exp()
+            tau = 1.0 / logit_scale
+        else:
+            logit_scale = 1.0 / self.temperature
+            tau = self.temperature
+
+        sim_q2g = logit_scale * query @ gallery.t()
+        sim_g2q = sim_q2g.t()
+
+        eps = self._compute_eps(positive_weights, query.shape[0])
+
+        loss_q2g = self._weighted_loss(sim_q2g, eps)
+        loss_g2q = self._weighted_loss(sim_g2q, eps)
+        total = (loss_q2g + loss_g2q) / 2
+
+        gate_q = embeddings.get("gate_q", 1.0)
+        gate_g = embeddings.get("gate_g", 1.0)
+
+        return {
+            "total": total,
+            "temperature": tau if isinstance(tau, torch.Tensor) else torch.tensor(tau, device=total.device),
+            "gate_q": torch.tensor(gate_q, device=total.device) if not isinstance(gate_q, torch.Tensor) else gate_q.detach(),
+            "gate_g": torch.tensor(gate_g, device=total.device) if not isinstance(gate_g, torch.Tensor) else gate_g.detach(),
+        }
--- a/vendor_reference/caption_test/src/models/asymmetric_encoder.py
+++ b/vendor_reference/caption_test/src/models/asymmetric_encoder.py
@@ -0,0 +1,656 @@
+from __future__ import annotations
+
+"""Asymmetric dual encoder for CVGL caption test on GTA-UAV.
+
+Architecture:
+  Query:   DINOv3 ViT-L/16 (LVD, frozen) + LRSCLIP text (L1/L2/L3) -> GatedFusion -> query
+  Gallery: DINOv3 ViT-L/16 (SAT, frozen) -> gallery
+  Loss:    InfoNCE(query, gallery)
+
+DINOv3 checkpoints use a custom key layout (not HuggingFace transformers).
+LRSCLIP (DGTRS-CLIP ViT-L-14) uses open_clip layout with KPS positional embeddings.
+"""
+
+import logging
+import math
+import warnings
+from pathlib import Path
+
+import coloredlogs
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+LOGGER = logging.getLogger("caption_test.model")
+coloredlogs.install(level="INFO", logger=LOGGER, fmt="%(asctime)s %(name)s %(levelname)s %(message)s")
+from safetensors.torch import load_file as load_safetensors
+
+from src.models.adapters import inject_lora_into_dgtrs, inject_mona_into_dinov3
+from src.models.dgtrs.model import DGTRSTextEncoder, load_dgtrs_text_encoder, tokenize_dgtrs
+from src.models.dual_encoder import GatedFusion, ProjectionHead
+from src.models.stripnet import inject_conv_mona_into_stripnet
+from src.models.stripnet_encoder import StripNetEncoder
+
+
+# ---------------------------------------------------------------------------
+# DINOv3 ViT-L/16 — minimal implementation matching checkpoint key layout
+# ---------------------------------------------------------------------------
+
+class DINOv3Attention(nn.Module):
+    """Multi-head self-attention with separate Q/K/V projections."""
+
+    def __init__(self, dim: int = 1024, num_heads: int = 16) -> None:
+        super().__init__()
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.scale = self.head_dim ** -0.5
+        self.q_proj = nn.Linear(dim, dim)
+        self.k_proj = nn.Linear(dim, dim, bias=False)
+        self.v_proj = nn.Linear(dim, dim)
+        self.o_proj = nn.Linear(dim, dim)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        B, N, C = x.shape
+        q = self.q_proj(x).reshape(B, N, self.num_heads, self.head_dim).permute(0, 2, 1, 3)
+        k = self.k_proj(x).reshape(B, N, self.num_heads, self.head_dim).permute(0, 2, 1, 3)
+        v = self.v_proj(x).reshape(B, N, self.num_heads, self.head_dim).permute(0, 2, 1, 3)
+        attn = F.scaled_dot_product_attention(q, k, v)
+        x = attn.permute(0, 2, 1, 3).reshape(B, N, C)
+        return self.o_proj(x)
+
+
+class DINOv3LayerScale(nn.Module):
+    """Per-channel learnable scale (lambda)."""
+
+    def __init__(self, dim: int) -> None:
+        super().__init__()
+        self.lambda1 = nn.Parameter(torch.ones(dim))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return x * self.lambda1
+
+
+class DINOv3MLP(nn.Module):
+    """SwiGLU-like MLP: up_proj + GELU + down_proj."""
+
+    def __init__(self, dim: int = 1024, mlp_dim: int = 4096) -> None:
+        super().__init__()
+        self.up_proj = nn.Linear(dim, mlp_dim)
+        self.down_proj = nn.Linear(mlp_dim, dim)
+        self.act = nn.GELU()
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.down_proj(self.act(self.up_proj(x)))
+
+
+class DINOv3Block(nn.Module):
+    """Single DINOv3 transformer block."""
+
+    def __init__(self, dim: int = 1024, num_heads: int = 16, mlp_dim: int = 4096) -> None:
+        super().__init__()
+        self.norm1 = nn.LayerNorm(dim)
+        self.attention = DINOv3Attention(dim, num_heads)
+        self.layer_scale1 = DINOv3LayerScale(dim)
+        self.norm2 = nn.LayerNorm(dim)
+        self.mlp = DINOv3MLP(dim, mlp_dim)
+        self.layer_scale2 = DINOv3LayerScale(dim)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = x + self.layer_scale1(self.attention(self.norm1(x)))
+        x = x + self.layer_scale2(self.mlp(self.norm2(x)))
+        return x
+
+
+class DINOv3Embeddings(nn.Module):
+    """Patch embedding + CLS token + register tokens."""
+
+    def __init__(
+        self,
+        dim: int = 1024,
+        patch_size: int = 16,
+        num_registers: int = 4,
+    ) -> None:
+        super().__init__()
+        self.patch_embeddings = nn.Conv2d(3, dim, patch_size, patch_size)
+        self.cls_token = nn.Parameter(torch.zeros(1, 1, dim))
+        self.register_tokens = nn.Parameter(torch.zeros(1, num_registers, dim))
+        self.mask_token = nn.Parameter(torch.zeros(1, 1, dim))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        B = x.shape[0]
+        patches = self.patch_embeddings(x).flatten(2).transpose(1, 2)  # [B, N, D]
+        N = patches.shape[1]
+
+        cls = self.cls_token.expand(B, -1, -1)
+        reg = self.register_tokens.expand(B, -1, -1)
+
+        # DINOv3: [CLS, registers, patches]
+        x = torch.cat([cls, reg, patches], dim=1)
+
+        # Positional embedding: interpolated sincos (RoPE applied in attention
+        # in original, but pretrained checkpoints bake it into weights).
+        # We use a simple learned-style pos embed computed on the fly.
+        pos = self._get_pos_embed(N, x.device, x.dtype)
+        # pos covers patches only, skip CLS + registers
+        x[:, 1 + reg.shape[1]:] = x[:, 1 + reg.shape[1]:] + pos
+
+        return x
+
+    def _get_pos_embed(self, n_patches: int, device: torch.device, dtype: torch.dtype) -> torch.Tensor:
+        # DINOv3 uses RoPE internally — no additive pos embed needed.
+        # Return zeros as placeholder (weights handle positioning via RoPE).
+        return torch.zeros(1, n_patches, self.cls_token.shape[-1], device=device, dtype=dtype)
+
+
+class DINOv3ViT(nn.Module):
+    """DINOv3 ViT-L/16 matching the checkpoint key layout.
+
+    Checkpoint keys:
+        embeddings.cls_token, embeddings.patch_embeddings.{weight,bias},
+        embeddings.register_tokens, embeddings.mask_token,
+        layer.{i}.attention.{q,k,v,o}_proj.{weight,bias},
+        layer.{i}.layer_scale{1,2}.lambda1,
+        layer.{i}.mlp.{up,down}_proj.{weight,bias},
+        layer.{i}.norm{1,2}.{weight,bias},
+        norm.{weight,bias}
+    """
+
+    def __init__(
+        self,
+        dim: int = 1024,
+        num_heads: int = 16,
+        mlp_dim: int = 4096,
+        num_layers: int = 24,
+        patch_size: int = 16,
+        num_registers: int = 4,
+    ) -> None:
+        super().__init__()
+        self.embeddings = DINOv3Embeddings(dim, patch_size, num_registers)
+        self.layer = nn.ModuleList([
+            DINOv3Block(dim, num_heads, mlp_dim) for _ in range(num_layers)
+        ])
+        self.norm = nn.LayerNorm(dim)
+        self.embed_dim = dim
+        self.gradient_checkpointing = False
+
+    def set_gradient_checkpointing(self, enable: bool = True) -> None:
+        """Enable/disable gradient checkpointing to trade compute for VRAM."""
+        self.gradient_checkpointing = enable
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """Forward pass. Returns CLS token embedding [B, dim]."""
+        x = self.embeddings(x)
+        for block in self.layer:
+            if self.gradient_checkpointing and self.training:
+                x = torch.utils.checkpoint.checkpoint(
+                    block, x, use_reentrant=False,
+                )
+            else:
+                x = block(x)
+        x = self.norm(x)
+        return x[:, 0]  # CLS token
+
+    @classmethod
+    def from_pretrained(cls, path: str | Path) -> DINOv3ViT:
+        """Load from .pth or .safetensors checkpoint."""
+        model = cls()
+        path = Path(path)
+        LOGGER.info("🧊 Loading DINOv3 from %s", path.name)
+        if path.suffix == ".safetensors":
+            state = load_safetensors(str(path))
+        else:
+            state = torch.load(str(path), map_location="cpu", weights_only=False)
+            if "model" in state:
+                state = state["model"]
+            elif "state_dict" in state:
+                state = state["state_dict"]
+        model.load_state_dict(state, strict=False)
+        n_params = sum(p.numel() for p in model.parameters())
+        LOGGER.info("🧊 DINOv3 loaded: %s params", f"{n_params:,}")
+        return model
+
+
+# LRSCLIPTextEncoder removed — replaced by official DGTRS architecture
+# in src/models/dgtrs/model.py (DGTRSTextEncoder)
+
+
+# ---------------------------------------------------------------------------
+# Text fusion MLP: concat L1/L2/L3 -> project to D
+# ---------------------------------------------------------------------------
+
+class TextFusionMLP(nn.Module):
+    """Fuse L1/L2/L3 text embeddings via concat + MLP.
+
+    [B, 3*text_dim] -> [B, out_dim]
+    """
+
+    def __init__(
+        self,
+        text_dim: int = 768,
+        out_dim: int = 1024,
+    ) -> None:
+        super().__init__()
+        self.mlp = nn.Sequential(
+            nn.Linear(3 * text_dim, out_dim),
+            nn.GELU(),
+            nn.Linear(out_dim, out_dim),
+        )
+
+    def forward(
+        self,
+        z_l1: torch.Tensor,
+        z_l2: torch.Tensor,
+        z_l3: torch.Tensor,
+    ) -> torch.Tensor:
+        """Fuse three text embeddings.
+
+        Args:
+            z_l1: L1 overview [B, text_dim].
+            z_l2: L2 full description [B, text_dim].
+            z_l3: L3 fingerprint [B, text_dim].
+
+        Returns:
+            Fused text embedding [B, out_dim].
+        """
+        cat = torch.cat([z_l1, z_l2, z_l3], dim=-1)
+        return self.mlp(cat)
+
+
+# ---------------------------------------------------------------------------
+# Main model: AsymmetricEncoder
+# ---------------------------------------------------------------------------
+
+# ResidualGateFusin experiment
+from .residual_fusions import ResidualGateType, GatedFusionResidual
+
+class AsymmetricEncoder(nn.Module):
+    """Dual encoder for CVGL with text fusion on both branches.
+
+    Supports two modes:
+      - **shared** (default): single DINOv3 WEB encoder for both drone and satellite,
+        one set of MONA adapters. Saves ~4-5 GB VRAM and halves adapter params.
+      - **asymmetric**: separate DINOv3 encoders (LVD for drone, SAT for satellite),
+        each with their own MONA adapters (legacy mode).
+
+    Query branch:   DINOv3 (drone) + text(L1/L2/L3) -> GatedFusion_q -> query [1024]
+    Gallery branch: DINOv3 (sat)   + text(L1/L2/L3) -> GatedFusion_g -> gallery [1024]
+
+    No projection layers — retrieval space is DINOv3 native 1024-dim.
+    Text fusion MLP is shared between branches (same caption format).
+    Two separate GatedFusion gates (drone/sat may weight text differently).
+
+    For satellite images without captions, GatedFusion passes image features through
+    (text_feat=None → gate acts as identity).
+
+    Args:
+        dino_web_path: Path to DINOv3 LVD checkpoint (used for both branches in shared mode).
+        dino_sat_path: Path to DINOv3 SAT checkpoint (only used in asymmetric mode).
+        lrsclip_path: Path to DGTRS-CLIP checkpoint (text encoder).
+        init_gate: Initial fusion gate (image weight).
+        baseline_mode: If True, gate = 1.0 (text ignored), DGTRS not loaded.
+        shared_encoder: If True, use single DINOv3 WEB for both branches.
+        device: Torch device string.
+    """
+
+    DINO_DIM = 1024
+    TEXT_DIM = 768
+
+    def __init__(
+        self,
+        # !!! ---------------------------------------------------------------
+        gate_type: ResidualGateType = ResidualGateType.simple_residual_one_gate,
+        dino_web_path: str = "nn_models/DINO_WEB/dinov3-vitl16-pretrain-lvd1689m.pth",
+        dino_sat_path: str = "nn_models/DINO_SAT/model.safetensors",
+        lrsclip_path: str = "nn_models/LRSCLIP/DGTRS-CLIP-ViT-L-14.pt",
+        init_gate: float = 0.7,
+        baseline_mode: bool = False,
+        shared_encoder: bool = False,
+        mona_bottleneck: int = 64,
+        mona_last_n_blocks: int = 24,
+        lora_rank: int = 4,
+        device: str = "cuda",
+        backbone: str = "dinov3",
+        stripnet_path: str = "nn_models/STRIPNET/stripnet_s.pth",
+        stripnet_mona_last_n_stages: int = 2,
+        stripnet_freeze: bool = True,
+    ) -> None:
+        super().__init__()
+        self.embed_dim = self.DINO_DIM  # native 1024 (StripNet projects 512 -> 1024)
+        self.baseline_mode = baseline_mode
+        self.shared_encoder = shared_encoder
+        self.backbone = backbone
+        self.device = device
+
+        # Image encoder(s) (frozen + MONA adapters).
+        if backbone == "stripnet":
+            # StripNet always operates as shared encoder (one CNN for both branches).
+            self.shared_encoder = True
+            self.image_encoder = StripNetEncoder(checkpoint_path=stripnet_path, out_dim=self.DINO_DIM)
+            if stripnet_freeze:
+                self._freeze(self.image_encoder.backbone)
+                LOGGER.info("StripNet backbone: frozen (Conv-MONA + projection trainable)")
+            else:
+                LOGGER.info("StripNet backbone: UNFROZEN — full fine-tune (use lower lr factor)")
+            if stripnet_mona_last_n_stages > 0:
+                inject_conv_mona_into_stripnet(
+                    self.image_encoder.backbone,
+                    bottleneck=mona_bottleneck,
+                    last_n_stages=stripnet_mona_last_n_stages,
+                )
+            else:
+                LOGGER.info("Conv-MONA disabled (stripnet_mona_last_n_stages=0)")
+            LOGGER.info("StripNet backbone: shared encoder, projection 512 -> %d", self.DINO_DIM)
+        elif shared_encoder:
+            self.image_encoder = DINOv3ViT.from_pretrained(dino_web_path)
+            self._freeze(self.image_encoder)
+            inject_mona_into_dinov3(self.image_encoder, bottleneck=mona_bottleneck, last_n_blocks=mona_last_n_blocks)
+            LOGGER.info("Shared encoder mode: single DINOv3 WEB for drone + satellite")
+        else:
+            self.drone_encoder = DINOv3ViT.from_pretrained(dino_web_path)
+            self.sat_encoder = DINOv3ViT.from_pretrained(dino_sat_path)
+            self._freeze(self.drone_encoder)
+            self._freeze(self.sat_encoder)
+            inject_mona_into_dinov3(self.drone_encoder, bottleneck=mona_bottleneck, last_n_blocks=mona_last_n_blocks)
+            inject_mona_into_dinov3(self.sat_encoder, bottleneck=mona_bottleneck, last_n_blocks=mona_last_n_blocks)
+            LOGGER.info("Asymmetric encoder mode: DINOv3 WEB (drone) + DINOv3 SAT (satellite)")
+
+        # Text encoder — official DGTRS architecture (frozen + LoRA).
+        if not baseline_mode:
+            self.text_encoder = load_dgtrs_text_encoder(lrsclip_path)
+            self._freeze(self.text_encoder)
+            inject_lora_into_dgtrs(self.text_encoder, rank=lora_rank)
+        else:
+            self.text_encoder = None
+
+        # Shared text fusion MLP: 3×768 -> 1024 (native DINOv3 dim).
+        if not baseline_mode:
+            self.text_fusion = TextFusionMLP(
+                text_dim=self.TEXT_DIM,
+                out_dim=self.DINO_DIM,
+            )
+
+        # Separate gated fusion for query and gallery branches.
+        #! Experimental Gated fusion on query branch.
+        self.fusion_query = GatedFusionResidual(gate_type=gate_type,
+            init_gate=init_gate, baseline_mode=baseline_mode)
+        self.fusion_gallery = GatedFusionResidual(gate_type=gate_type,
+            init_gate=init_gate, baseline_mode=baseline_mode)
+
+
+    @staticmethod
+    def _freeze(module: nn.Module) -> None:
+        for p in module.parameters():
+            p.requires_grad = False
+        module.eval()
+
+    def encode_drone(self, images: torch.Tensor) -> torch.Tensor:
+        """Encode drone images with MONA adapters. Returns [B, 1024]."""
+        if self.shared_encoder:
+            return self.image_encoder(images)
+        return self.drone_encoder(images)
+
+    def encode_satellite(self, images: torch.Tensor) -> torch.Tensor:
+        """Encode satellite images with MONA adapters. Returns [B, 1024]."""
+        if self.shared_encoder:
+            return self.image_encoder(images)
+        return self.sat_encoder(images)
+
+    def encode_text_levels(
+        self,
+        l1_texts: list[str],
+        l2_texts: list[str],
+        l3_texts: list[str],
+    ) -> torch.Tensor | None:
+        """Encode L1/L2/L3 captions and fuse. Returns [B, 1024] or None.
+
+        Returns None if all captions are empty (no text available).
+        For mixed batches (some have captions, some don't), encodes all
+        texts (empty strings tokenize to pad+EOS — their outputs must be
+        masked downstream, see `_fuse_with_mask`).
+        """
+        # Check if any caption is non-empty.
+        if all(t == "" for t in l1_texts):
+            return None
+
+        z_l1 = self._encode_single_text(l1_texts)
+        z_l2 = self._encode_single_text(l2_texts)
+        z_l3 = self._encode_single_text(l3_texts)
+        fused = self.text_fusion(z_l1, z_l2, z_l3)
+        return F.normalize(fused, dim=-1)
+
+    def _encode_single_text(self, texts: list[str]) -> torch.Tensor:
+        """Tokenize and encode a list of strings using DGTRS tokenizer."""
+        tokens = tokenize_dgtrs(list(texts)).to(self.device)
+        return self.text_encoder(tokens)
+
+    def _fuse_with_mask(
+        self,
+        img_feat: torch.Tensor,
+        l1_texts: list[str] | None,
+        l2_texts: list[str] | None,
+        l3_texts: list[str] | None,
+        fusion: GatedFusionResidual,
+    ) -> torch.Tensor:
+        """Fuse image features with optional text, respecting per-sample presence.
+
+        For samples where caption is an empty string, output falls back to
+        pure image features (avoiding noise contamination from empty-string
+        text embeddings). For samples with captions, applies the standard
+        gated fusion `σ(α)·img + (1-σ(α))·text`.
+
+        Returns L2-normalized [B, D] embedding.
+        """
+        if (
+            self.baseline_mode
+            or l1_texts is None
+            or l2_texts is None
+            or l3_texts is None
+        ):
+            return F.normalize(fusion(img_feat, None), dim=-1)
+
+        has_text = torch.tensor(
+            [t != "" for t in l1_texts], dtype=torch.bool, device=img_feat.device,
+        )
+        if not has_text.any():
+            return F.normalize(fusion(img_feat, None), dim=-1)
+
+        z_text = self.encode_text_levels(l1_texts, l2_texts, l3_texts)
+        if z_text is None:
+            return F.normalize(fusion(img_feat, None), dim=-1)
+
+        # Per-sample fusion: text-present samples use full gated fusion,
+        # empty-caption samples pass through pure image features.
+        fused_with_text = fusion(img_feat, z_text)
+        out = torch.where(has_text.unsqueeze(-1), fused_with_text, img_feat)
+        return F.normalize(out, dim=-1)
+
+    def encode_query(
+        self,
+        drone_img: torch.Tensor,
+        caption_l1: list[str] | None = None,
+        caption_l2: list[str] | None = None,
+        caption_l3: list[str] | None = None,
+    ) -> torch.Tensor:
+        """Encode drone → normalized query embedding with per-sample text mask."""
+        drone_feat = self.encode_drone(drone_img)
+        return self._fuse_with_mask(
+            drone_feat, caption_l1, caption_l2, caption_l3, self.fusion_query,
+        )
+
+    def encode_gallery(
+        self,
+        sat_img: torch.Tensor,
+        sat_caption_l1: list[str] | None = None,
+        sat_caption_l2: list[str] | None = None,
+        sat_caption_l3: list[str] | None = None,
+    ) -> torch.Tensor:
+        """Encode satellite → normalized gallery embedding with per-sample text mask."""
+        sat_feat = self.encode_satellite(sat_img)
+        return self._fuse_with_mask(
+            sat_feat, sat_caption_l1, sat_caption_l2, sat_caption_l3, self.fusion_gallery,
+        )
+
+    def forward(
+        self,
+        drone_img: torch.Tensor,
+        sat_img: torch.Tensor,
+        caption_l1: list[str] | None = None,
+        caption_l2: list[str] | None = None,
+        caption_l3: list[str] | None = None,
+        sat_caption_l1: list[str] | None = None,
+        sat_caption_l2: list[str] | None = None,
+        sat_caption_l3: list[str] | None = None,
+    ) -> dict[str, torch.Tensor]:
+        """Forward pass.
+
+        Both branches use per-sample caption masking: samples with an empty
+        caption string fall back to pure image features instead of being
+        fused with noise from empty-string text embeddings.
+
+        Args:
+            drone_img: Drone images [B, 3, 256, 256].
+            sat_img: Satellite images [B, 3, 256, 256].
+            caption_l1/l2/l3: Drone L1/L2/L3 captions.
+            sat_caption_l1/l2/l3: Satellite L1/L2/L3 captions.
+
+        Returns:
+            Dict with 'query' [B, embed_dim], 'gallery' [B, embed_dim],
+            'gate_q', 'gate_g'.
+        """
+        query = self.encode_query(drone_img, caption_l1, caption_l2, caption_l3)
+        gallery = self.encode_gallery(sat_img, sat_caption_l1, sat_caption_l2, sat_caption_l3)
+        return {
+            "query": query,
+            "gallery": gallery,
+            "gate_q": self.fusion_query.gate_value,
+            "gate_g": self.fusion_gallery.gate_value,
+        }
+
+    def trainable_parameters(self) -> list[nn.Parameter]:
+        """Return list of parameters that require grad."""
+        return [p for p in self.parameters() if p.requires_grad]
+
+    def save_checkpoint(self, path: str | Path, **extra) -> None:
+        """Save model checkpoint with metadata."""
+        path = Path(path)
+        path.parent.mkdir(parents=True, exist_ok=True)
+        ckpt = {
+            "model_state": self.state_dict(),
+            "baseline_mode": self.baseline_mode,
+            "shared_encoder": self.shared_encoder,
+            **extra,
+        }
+        tmp = path.with_suffix(path.suffix + ".tmp")
+        torch.save(ckpt, tmp)
+        tmp.replace(path)
+        LOGGER.info("💾 Model saved to %s", path)
+
+    @classmethod
+    def load_checkpoint(
+        cls,
+        path: str | Path,
+        dino_web_path: str = "nn_models/DINO_WEB/dinov3-vitl16-pretrain-lvd1689m.pth",
+        dino_sat_path: str = "nn_models/DINO_SAT/model.safetensors",
+        lrsclip_path: str = "nn_models/LRSCLIP/DGTRS-CLIP-ViT-L-14.pt",
+        device: str = "cuda",
+    ) -> tuple[AsymmetricEncoder, dict]:
+        """Load model from checkpoint.
+
+        First builds the model (loading frozen backbones), then loads
+        the saved trainable weights on top.
+
+        Returns:
+            (model, checkpoint_dict) — model ready for eval/resume,
+            checkpoint_dict has optimizer_state, epoch, etc.
+        """
+        path = Path(path)
+        LOGGER.info("📂 Loading checkpoint from %s", path)
+        ckpt = torch.load(str(path), map_location="cpu", weights_only=False)
+
+        model = cls(
+            dino_web_path=dino_web_path,
+            dino_sat_path=dino_sat_path,
+            lrsclip_path=lrsclip_path,
+            baseline_mode=ckpt.get("baseline_mode", False),
+            shared_encoder=ckpt.get("shared_encoder", False),
+            mona_bottleneck=ckpt.get("mona_bottleneck", 64),
+            mona_last_n_blocks=ckpt.get("mona_last_n_blocks", 24),
+            device=device,
+        )
+        model.load_state_dict(ckpt["model_state"], strict=False)
+        model = model.to(device)
+        LOGGER.info("✅ Checkpoint loaded (epoch=%s)", ckpt.get("epoch", "?"))
+        return model, ckpt
+
+    def train(self, mode: bool = True) -> AsymmetricEncoder:
+        """Override to keep frozen encoders in eval mode."""
+        super().train(mode)
+        if self.shared_encoder:
+            self.image_encoder.eval()
+        else:
+            self.drone_encoder.eval()
+            self.sat_encoder.eval()
+        if self.text_encoder is not None:
+            self.text_encoder.train(mode)
+        return self
+
+
+# ---------------------------------------------------------------------------
+# Image preprocessing (DINOv3: 256x256, ImageNet normalization)
+# ---------------------------------------------------------------------------
+
+_IMAGENET_MEAN = [0.485, 0.456, 0.406]
+_IMAGENET_STD = [0.229, 0.224, 0.225]
+
+
+def get_dino_transform(image_size: int = 256) -> torch.nn.Module:
+    """Build eval/inference image transform for DINOv3 input."""
+    from torchvision import transforms
+    return transforms.Compose([
+        transforms.Resize(image_size, interpolation=transforms.InterpolationMode.BICUBIC),
+        transforms.CenterCrop(image_size),
+        transforms.ToTensor(),
+        transforms.Normalize(mean=_IMAGENET_MEAN, std=_IMAGENET_STD),
+    ])
+
+
+def get_drone_train_transform(image_size: int = 256) -> torch.nn.Module:
+    """Build training augmentation for drone images.
+
+    Includes: RandomResizedCrop, HFlip, rotation, color jitter,
+    grayscale, Gaussian blur.
+    """
+    from torchvision import transforms
+    return transforms.Compose([
+        transforms.RandomResizedCrop(
+            image_size, scale=(0.7, 1.0),
+            interpolation=transforms.InterpolationMode.BICUBIC,
+        ),
+        transforms.RandomHorizontalFlip(p=0.5),
+        transforms.RandomRotation(degrees=15),
+        transforms.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.2, hue=0.1),
+        transforms.RandomGrayscale(p=0.05),
+        transforms.GaussianBlur(kernel_size=3, sigma=(0.1, 2.0)),
+        transforms.ToTensor(),
+        transforms.Normalize(mean=_IMAGENET_MEAN, std=_IMAGENET_STD),
+    ])
+
+
+def get_satellite_train_transform(image_size: int = 256) -> torch.nn.Module:
+    """Build training augmentation for satellite images.
+
+    Lighter than drone: no rotation or blur (satellite is nadir/consistent).
+    Includes: RandomResizedCrop, HFlip, color jitter, grayscale.
+    """
+    from torchvision import transforms
+    return transforms.Compose([
+        transforms.RandomResizedCrop(
+            image_size, scale=(0.7, 1.0),
+            interpolation=transforms.InterpolationMode.BICUBIC,
+        ),
+        transforms.RandomHorizontalFlip(p=0.5),
+        transforms.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.2, hue=0.1),
+        transforms.RandomGrayscale(p=0.05),
+        transforms.ToTensor(),
+        transforms.Normalize(mean=_IMAGENET_MEAN, std=_IMAGENET_STD),
+    ])
--- a/vendor_reference/caption_test/src/models/residual_fusions.py
+++ b/vendor_reference/caption_test/src/models/residual_fusions.py
@@ -0,0 +1,154 @@
+import torch
+import torch.nn as nn
+import gin
+
+#! GATE-FUSIONS MODIFICATIONS ---------------------------------
+#! in_dim = 1024
+
+from enum import Enum
+import math
+
+class ResidualGateType(Enum):
+    simple_residual_one_gate = 0,
+    cross_gate = 1,
+    gate_sum = 2,
+    alpha_res_cat = 3,
+    alpha_res_sum = 4
+
+# TODO: add GatedFusionresidual class to gin
+
+def init_bias_for_sigmoid(linear: nn.Linear, value: float) -> None:
+    nn.init.zeros_(linear.weight)
+    nn.init.constant_(linear.bias, math.log(value / (1.0 - value)))
+
+def init_residual_projs(linear: nn.Linear, scale: float) -> None:
+    nn.init.xavier_uniform_(linear.weight, gain=scale)
+    nn.init.zeros_(linear.bias)
+
+RESIDUAL_GATES = {
+    ResidualGateType.alpha_res_sum,
+    ResidualGateType.alpha_res_cat
+}
+
+@gin.configurable
+class GatedFusionResidual(nn.Module):
+    """Learnable gated fusion of image and text embeddings.
+
+    V1 - Simple residual gating with 1 common gate: 
+    V2 - Cross residual gating with 2 cross-gates
+    V3 - Gate + Simple Sum of feats x & y
+    V4 - Alpha-weighted residual sum (per sample)
+    V5 - Alpha-weighted residual concat (per sample)
+    """
+
+    def __init__(self, gate_type: ResidualGateType, 
+                 init_gate: float = 0.7, in_dim = 1024, 
+                 init_res_weight: float = 0.1, residual_proj_scale: float = 0.1,  
+                 baseline_mode: bool = False, 
+                 ) -> None:
+        super().__init__()
+        # alpha is in logit space: sigmoid(alpha) = init_gate
+        init_alpha = torch.log(torch.tensor(init_gate / (1.0 - init_gate)))
+        self.alpha = nn.Parameter(init_alpha)
+        
+        # alphas for separated cases
+        if gate_type == ResidualGateType.cross_gate:
+            init_alpha_img_cross_gate = torch.log(torch.tensor(init_gate / (1.0 - init_gate)))
+            init_alpha_text_cross_gate = torch.log(torch.tensor(init_gate / (1.0 - init_gate)))
+            self.alpha_img = nn.Parameter(init_alpha_img_cross_gate)
+            self.alpha_text = nn.Parameter(init_alpha_text_cross_gate)
+        
+        # weight for sum and cat residual 
+        if gate_type in RESIDUAL_GATES:
+            self.final_cat_residual_proj = nn.Linear(in_dim * 2, in_dim)
+            self.weight_net_for_sum = nn.Linear(in_dim, 1)
+            self.weight_net_for_cat = nn.Linear(in_dim * 2, 1)
+            init_bias_for_sigmoid(self.weight_net_for_sum, value=init_res_weight)
+            init_bias_for_sigmoid(self.weight_net_for_cat, value=init_res_weight)
+            init_residual_projs(self.final_cat_residual_proj, scale=residual_proj_scale)
+
+        self.gate_type = gate_type
+        self.baseline_mode = baseline_mode
+
+    def FuseSRGF(self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        gate = torch.sigmoid(self.alpha)
+        img_res = img_feat * gate + img_feat
+        text_res = text_feat * (1 - gate) + text_feat
+        fused_vec = img_res + text_res
+        return fused_vec
+    
+    def FuseRCGF(self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        gate_img = torch.sigmoid(self.alpha_img)
+        gate_text = torch.sigmoid(self.alpha_text)
+        z_img = img_feat + gate_text * img_feat
+        z_text = text_feat + gate_img * text_feat
+        fused_vec = z_img + z_text
+        return fused_vec
+    
+    def FuseGSUM(self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        gate = torch.sigmoid(self.alpha)
+        fuzed_vec = img_feat + text_feat + gate * img_feat + (1.0 - gate) * text_feat
+        return fuzed_vec
+    
+    def FuseARGFSum(
+        self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        gate = torch.sigmoid(self.alpha)
+        residual = img_feat + text_feat
+        res_weight = torch.sigmoid(self.weight_net_for_sum(residual))
+        fuzed_vec = gate * img_feat + (1.0 - gate) * text_feat + res_weight * residual
+        return fuzed_vec
+
+    def FuseARGFCat(
+        self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        gate = torch.sigmoid(self.alpha)
+        cat_vec = torch.cat([img_feat, text_feat], dim=-1)
+        residual = self.final_cat_residual_proj(cat_vec)
+        res_weight = torch.sigmoid(self.weight_net_for_cat(cat_vec))
+        fuzed_vec = gate * img_feat + (1.0 - gate) * text_feat + res_weight * residual
+        return fuzed_vec
+
+    def forward(
+        self,
+        img_feat: torch.Tensor,
+        text_feat: torch.Tensor | None,
+    ) -> torch.Tensor:
+        if text_feat is None or self.baseline_mode:
+            return img_feat
+
+        if self.gate_type == ResidualGateType.simple_residual_one_gate:
+            fused_vec = self.FuseSRGF(img_feat=img_feat, text_feat=text_feat)
+        if self.gate_type == ResidualGateType.cross_gate:
+            fused_vec = self.FuseRCGF(img_feat=img_feat, text_feat=text_feat)
+        if self.gate_type == ResidualGateType.gate_sum:
+            fused_vec = self.FuseGSUM(img_feat=img_feat, text_feat=text_feat)
+        if self.gate_type == ResidualGateType.alpha_res_sum:
+            fused_vec = self.FuseARGFSum(img_feat=img_feat, text_feat=text_feat)
+        if self.gate_type == ResidualGateType.alpha_res_cat:
+            fused_vec = self.FuseARGFCat(img_feat=img_feat, text_feat=text_feat)
+
+        return fused_vec
+    
+    @property
+    def gate_value(self) -> float:
+        """Current gate value (image weight). 1.0 = text ignored."""
+        if self.baseline_mode:
+            return 1.0
+        return torch.sigmoid(self.alpha).item()
+    
+
+    
--- a/vendor_reference/caption_test/src/models/stripnet/conv_mona.py
+++ b/vendor_reference/caption_test/src/models/stripnet/conv_mona.py
@@ -0,0 +1,106 @@
+"""Conv-MONA: 2D adaptation of MONA (CVPR 2025) for hierarchical CNN backbones.
+
+MONA paper applies sequence-form adapters after MSA / MLP in ViT blocks. Here we
+mirror that idea in [B, C, H, W] form: BN → 1×1 Down(C→bn) → multi-scale DWConv
+{3,5,7} mean → +residual → GELU → 1×1 Up(bn→C). Layer scale (γ) channel-wise,
+init 1e-6 for near-identity start. Two adapters per StripNet Block: post-attn
+and post-mlp.
+"""
+from __future__ import annotations
+
+import logging
+import torch
+import torch.nn as nn
+
+from src.models.stripnet.model import StripNet, Block
+
+LOGGER = logging.getLogger("caption_test.stripnet.adapters")
+
+
+class ConvMona(nn.Module):
+    """Single Conv-MONA adapter.
+
+    Args:
+        dim: input channel dim.
+        bottleneck: bottleneck channel dim (e.g. 64).
+        gamma_init: layer-scale init value (1e-6 for near-identity at start).
+    """
+
+    def __init__(self, dim: int, bottleneck: int = 64, gamma_init: float = 1e-6) -> None:
+        super().__init__()
+        self.norm = nn.BatchNorm2d(dim)
+        self.down = nn.Conv2d(dim, bottleneck, kernel_size=1, bias=True)
+        self.dw3 = nn.Conv2d(bottleneck, bottleneck, kernel_size=3, padding=1, groups=bottleneck, bias=True)
+        self.dw5 = nn.Conv2d(bottleneck, bottleneck, kernel_size=5, padding=2, groups=bottleneck, bias=True)
+        self.dw7 = nn.Conv2d(bottleneck, bottleneck, kernel_size=7, padding=3, groups=bottleneck, bias=True)
+        self.act = nn.GELU()
+        self.up = nn.Conv2d(bottleneck, dim, kernel_size=1, bias=True)
+        # Channel-wise layer scale (γ), broadcast across H, W.
+        self.gamma = nn.Parameter(gamma_init * torch.ones(dim), requires_grad=True)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        h = self.norm(x)
+        h = self.down(h)
+        h = (self.dw3(h) + self.dw5(h) + self.dw7(h)) / 3.0 + h
+        h = self.act(h)
+        h = self.up(h)
+        return self.gamma.view(1, -1, 1, 1) * h
+
+
+def _patched_block_forward(block: Block, mona_attn: ConvMona, mona_mlp: ConvMona):
+    """Closure that wraps a Block.forward with two Conv-MONA residuals."""
+    orig_attn = block.attn
+    orig_mlp = block.mlp
+    orig_norm1 = block.norm1
+    orig_norm2 = block.norm2
+    orig_drop = block.drop_path
+    ls1 = block.layer_scale_1
+    ls2 = block.layer_scale_2
+
+    def forward(x: torch.Tensor) -> torch.Tensor:
+        x = x + orig_drop(ls1.unsqueeze(-1).unsqueeze(-1) * orig_attn(orig_norm1(x))) + mona_attn(x)
+        x = x + orig_drop(ls2.unsqueeze(-1).unsqueeze(-1) * orig_mlp(orig_norm2(x))) + mona_mlp(x)
+        return x
+
+    return forward
+
+
+def inject_conv_mona_into_stripnet(
+    model: StripNet,
+    bottleneck: int = 64,
+    last_n_stages: int = 2,
+    use_bf16: bool = False,
+) -> int:
+    """Inject Conv-MONA adapters into the deepest `last_n_stages` of StripNet.
+
+    Each Block in the targeted stages gets two adapters (post-attn, post-mlp).
+    Returns the number of adapters injected.
+
+    Stages are 1-indexed in StripNet (block1..block4). With `last_n_stages=2`
+    we adapt block3 and block4 — the deepest, semantically richest features.
+    """
+    n_stages = model.num_stages
+    target_stages = list(range(max(1, n_stages - last_n_stages + 1), n_stages + 1))
+    n_added = 0
+
+    for stage_idx in target_stages:
+        blocks: nn.ModuleList = getattr(model, f"block{stage_idx}")
+        dim = model.embed_dims[stage_idx - 1]
+        for blk_idx, block in enumerate(blocks):
+            mona_a = ConvMona(dim=dim, bottleneck=bottleneck)
+            mona_m = ConvMona(dim=dim, bottleneck=bottleneck)
+            if use_bf16:
+                mona_a.to(dtype=torch.bfloat16)
+                mona_m.to(dtype=torch.bfloat16)
+            # Register as submodules so they get moved by .to(device) / .train() etc.
+            block.add_module(f"mona_attn", mona_a)
+            block.add_module(f"mona_mlp", mona_m)
+            block.forward = _patched_block_forward(block, mona_a, mona_m)
+            n_added += 2
+
+    n_trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
+    LOGGER.info(
+        "🔧 Conv-MONA injected: %d adapters in stages %s, %d trainable params (bottleneck=%d)",
+        n_added, target_stages, n_trainable, bottleneck,
+    )
+    return n_added
--- a/vendor_reference/caption_test/src/models/stripnet/model.py
+++ b/vendor_reference/caption_test/src/models/stripnet/model.py
@@ -0,0 +1,253 @@
+"""StripNet (small) backbone — adapted from Strip-R-CNN (HVision-NKU).
+
+Self-contained: no external utils. State-dict naming follows the upstream
+ImageNet-pretrained checkpoint (`conv_spatial1/2` for the strip kernels).
+"""
+from __future__ import annotations
+
+import logging
+import math
+from functools import partial
+from pathlib import Path
+from typing import List
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+LOGGER = logging.getLogger("caption_test.stripnet")
+
+
+def _to_2tuple(x):
+    if isinstance(x, (tuple, list)):
+        return tuple(x)
+    return (x, x)
+
+
+def drop_path(x: torch.Tensor, drop_prob: float = 0.0, training: bool = False) -> torch.Tensor:
+    if drop_prob == 0.0 or not training:
+        return x
+    keep = 1.0 - drop_prob
+    shape = (x.shape[0],) + (1,) * (x.ndim - 1)
+    rand = x.new_empty(shape).bernoulli_(keep)
+    if keep > 0:
+        rand.div_(keep)
+    return x * rand
+
+
+class DropPath(nn.Module):
+    def __init__(self, p: float = 0.0) -> None:
+        super().__init__()
+        self.p = p
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return drop_path(x, self.p, self.training)
+
+
+class DWConv(nn.Module):
+    def __init__(self, dim: int) -> None:
+        super().__init__()
+        self.dwconv = nn.Conv2d(dim, dim, 3, 1, 1, bias=True, groups=dim)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.dwconv(x)
+
+
+class Mlp(nn.Module):
+    def __init__(self, in_features: int, hidden_features: int, drop: float = 0.0) -> None:
+        super().__init__()
+        self.fc1 = nn.Conv2d(in_features, hidden_features, 1)
+        self.dwconv = DWConv(hidden_features)
+        self.act = nn.GELU()
+        self.fc2 = nn.Conv2d(hidden_features, in_features, 1)
+        self.drop = nn.Dropout(drop)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.fc1(x)
+        x = self.dwconv(x)
+        x = self.act(x)
+        x = self.drop(x)
+        x = self.fc2(x)
+        x = self.drop(x)
+        return x
+
+
+class StripGatingUnit(nn.Module):
+    """Strip spatial gating: 5x5 DWConv -> (1, k2) -> (k2, 1) -> 1x1 -> gate."""
+
+    def __init__(self, dim: int, k1: int, k2: int) -> None:
+        super().__init__()
+        self.conv0 = nn.Conv2d(dim, dim, 5, padding=2, groups=dim)
+        # Names match upstream pretrained checkpoint: conv_spatial1 / conv_spatial2.
+        self.conv_spatial1 = nn.Conv2d(dim, dim, kernel_size=(k1, k2), stride=1,
+                                       padding=(k1 // 2, k2 // 2), groups=dim)
+        self.conv_spatial2 = nn.Conv2d(dim, dim, kernel_size=(k2, k1), stride=1,
+                                       padding=(k2 // 2, k1 // 2), groups=dim)
+        self.conv1 = nn.Conv2d(dim, dim, 1)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        attn = self.conv0(x)
+        attn = self.conv_spatial1(attn)
+        attn = self.conv_spatial2(attn)
+        attn = self.conv1(attn)
+        return x * attn
+
+
+class StripAttention(nn.Module):
+    def __init__(self, dim: int, k1: int, k2: int) -> None:
+        super().__init__()
+        self.proj_1 = nn.Conv2d(dim, dim, 1)
+        self.activation = nn.GELU()
+        self.spatial_gating_unit = StripGatingUnit(dim, k1, k2)
+        self.proj_2 = nn.Conv2d(dim, dim, 1)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        residual = x
+        x = self.proj_1(x)
+        x = self.activation(x)
+        x = self.spatial_gating_unit(x)
+        x = self.proj_2(x)
+        return x + residual
+
+
+class Block(nn.Module):
+    def __init__(self, dim: int, mlp_ratio: float, k1: int, k2: int, drop: float, drop_path: float) -> None:
+        super().__init__()
+        self.norm1 = nn.BatchNorm2d(dim)
+        self.norm2 = nn.BatchNorm2d(dim)
+        self.attn = StripAttention(dim, k1, k2)
+        self.drop_path = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+        self.mlp = Mlp(dim, int(dim * mlp_ratio), drop=drop)
+        ls_init = 1e-2
+        self.layer_scale_1 = nn.Parameter(ls_init * torch.ones(dim), requires_grad=True)
+        self.layer_scale_2 = nn.Parameter(ls_init * torch.ones(dim), requires_grad=True)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = x + self.drop_path(
+            self.layer_scale_1.unsqueeze(-1).unsqueeze(-1) * self.attn(self.norm1(x))
+        )
+        x = x + self.drop_path(
+            self.layer_scale_2.unsqueeze(-1).unsqueeze(-1) * self.mlp(self.norm2(x))
+        )
+        return x
+
+
+class OverlapPatchEmbed(nn.Module):
+    def __init__(self, patch_size: int, stride: int, in_chans: int, embed_dim: int) -> None:
+        super().__init__()
+        ph, pw = _to_2tuple(patch_size)
+        self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=(ph, pw), stride=stride,
+                              padding=(ph // 2, pw // 2))
+        self.norm = nn.BatchNorm2d(embed_dim)
+
+    def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, int, int]:
+        x = self.proj(x)
+        _, _, H, W = x.shape
+        x = self.norm(x)
+        return x, H, W
+
+
+class StripNet(nn.Module):
+    """Strip-R-CNN backbone: 4-stage hierarchical CNN with strip-shaped DWConv attention.
+
+    Output: list of [B, C_i, H/s_i, W/s_i] per stage. Use `forward_last_features` for
+    the deepest stage only.
+    """
+
+    def __init__(
+        self,
+        embed_dims: List[int] = [64, 128, 320, 512],
+        mlp_ratios: List[int] = [8, 8, 4, 4],
+        k1s: List[int] = [1, 1, 1, 1],
+        k2s: List[int] = [19, 19, 19, 19],
+        depths: List[int] = [2, 2, 4, 2],
+        drop_rate: float = 0.1,
+        drop_path_rate: float = 0.15,
+        in_chans: int = 3,
+    ) -> None:
+        super().__init__()
+        self.depths = depths
+        self.num_stages = len(depths)
+        self.embed_dims = embed_dims
+
+        dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))]
+        cur = 0
+        for i in range(self.num_stages):
+            patch_embed = OverlapPatchEmbed(
+                patch_size=7 if i == 0 else 3,
+                stride=4 if i == 0 else 2,
+                in_chans=in_chans if i == 0 else embed_dims[i - 1],
+                embed_dim=embed_dims[i],
+            )
+            block = nn.ModuleList([
+                Block(dim=embed_dims[i], mlp_ratio=mlp_ratios[i], k1=k1s[i], k2=k2s[i],
+                      drop=drop_rate, drop_path=dpr[cur + j])
+                for j in range(depths[i])
+            ])
+            norm = nn.LayerNorm(embed_dims[i], eps=1e-6)
+            cur += depths[i]
+            setattr(self, f"patch_embed{i + 1}", patch_embed)
+            setattr(self, f"block{i + 1}", block)
+            setattr(self, f"norm{i + 1}", norm)
+
+    def forward_features(self, x: torch.Tensor) -> List[torch.Tensor]:
+        B = x.shape[0]
+        outs: List[torch.Tensor] = []
+        for i in range(self.num_stages):
+            patch_embed = getattr(self, f"patch_embed{i + 1}")
+            block = getattr(self, f"block{i + 1}")
+            norm = getattr(self, f"norm{i + 1}")
+            x, H, W = patch_embed(x)
+            for blk in block:
+                x = blk(x)
+            x = x.flatten(2).transpose(1, 2)
+            x = norm(x)
+            x = x.reshape(B, H, W, -1).permute(0, 3, 1, 2).contiguous()
+            outs.append(x)
+        return outs
+
+    def forward_last_features(self, x: torch.Tensor) -> torch.Tensor:
+        return self.forward_features(x)[-1]
+
+
+def get_stripnet_small() -> StripNet:
+    return StripNet(
+        embed_dims=[64, 128, 320, 512],
+        mlp_ratios=[8, 8, 4, 4],
+        k1s=[1, 1, 1, 1],
+        k2s=[19, 19, 19, 19],
+        depths=[2, 2, 4, 2],
+        drop_rate=0.1,
+        drop_path_rate=0.15,
+    )
+
+
+def load_stripnet_small_pretrained(checkpoint_path: str | Path) -> StripNet:
+    """Build StripNet-small and load ImageNet-pretrained weights.
+
+    Strips the classification `head.*` keys. Tolerates missing/extra keys
+    (norm{N}.* are LayerNorm here vs BatchNorm in some forks — we keep LN).
+    """
+    LOGGER.info("📐 Loading StripNet-small from %s", checkpoint_path)
+    model = get_stripnet_small()
+    raw = torch.load(checkpoint_path, map_location="cpu", weights_only=False)
+    state = raw.get("state_dict", raw) if isinstance(raw, dict) else raw
+
+    # Drop classification head + the BN-form norm{N} keys if present (we use LN here).
+    drop_prefixes = ("head.",)
+    cleaned = {k: v for k, v in state.items() if not any(k.startswith(p) for p in drop_prefixes)}
+
+    # The pretrained checkpoint stores norm{N} as BatchNorm2d (running_mean/var/num_batches_tracked).
+    # Our code uses LayerNorm at this position. Strip BN running stats if found; copy weight/bias.
+    for n in (1, 2, 3, 4):
+        for suffix in ("running_mean", "running_var", "num_batches_tracked"):
+            cleaned.pop(f"norm{n}.{suffix}", None)
+
+    missing, unexpected = model.load_state_dict(cleaned, strict=False)
+    if missing:
+        LOGGER.info("StripNet missing keys (expected for newly-init layers): %d", len(missing))
+    if unexpected:
+        LOGGER.info("StripNet unexpected keys (ignored): %d", len(unexpected))
+    LOGGER.info("📐 StripNet-small loaded: %d params", sum(p.numel() for p in model.parameters()))
+    return model
--- a/vendor_reference/caption_test/src/models/stripnet_encoder.py
+++ b/vendor_reference/caption_test/src/models/stripnet_encoder.py
@@ -0,0 +1,48 @@
+"""StripNet image encoder wrapper for the caption-test pipeline.
+
+Exposes the same interface as DINOv3ViT: `forward(images) -> [B, embed_dim]`.
+StripNet's deepest stage produces [B, 512, H/32, W/32]; we apply global average
+pooling (GAP) and project to the target retrieval dimension via Linear(512→1024)
+to match DINOv3 native dim and keep TextFusionMLP unchanged.
+"""
+from __future__ import annotations
+
+import logging
+import torch
+import torch.nn as nn
+
+from src.models.stripnet import StripNet, load_stripnet_small_pretrained
+
+LOGGER = logging.getLogger("caption_test.stripnet_encoder")
+
+
+class StripNetEncoder(nn.Module):
+    """StripNet-small + GAP + projection to `out_dim`.
+
+    Frozen backbone (BatchNorm in eval mode); only the projection head and
+    any injected Conv-MONA adapters are trainable.
+    """
+
+    LAST_STAGE_DIM = 512  # StripNet-small last stage embed dim
+
+    def __init__(self, checkpoint_path: str, out_dim: int = 1024) -> None:
+        super().__init__()
+        self.out_dim = out_dim
+        self.backbone: StripNet = load_stripnet_small_pretrained(checkpoint_path)
+        self.pool = nn.AdaptiveAvgPool2d(1)
+        self.projection = nn.Linear(self.LAST_STAGE_DIM, out_dim)
+        nn.init.trunc_normal_(self.projection.weight, std=0.02)
+        nn.init.zeros_(self.projection.bias)
+
+    def train(self, mode: bool = True):
+        """Override: keep frozen backbone in eval mode (BN running stats stable)."""
+        super().train(mode)
+        # Frozen backbone always in eval; trainable adapters/projection follow `mode`.
+        if not any(p.requires_grad for p in self.backbone.parameters()):
+            self.backbone.eval()
+        return self
+
+    def forward(self, images: torch.Tensor) -> torch.Tensor:
+        feat = self.backbone.forward_last_features(images)  # [B, 512, H/32, W/32]
+        pooled = self.pool(feat).flatten(1)                  # [B, 512]
+        return self.projection(pooled)                        # [B, out_dim]
--- a/vendor_reference/caption_test/src/training/profiling.py
+++ b/vendor_reference/caption_test/src/training/profiling.py
@@ -0,0 +1,166 @@
+from __future__ import annotations
+
+"""PyTorch Profiler wrapper for training performance analysis.
+
+Profiles the first N batches of training to identify bottlenecks
+in CUDA/CPU execution, memory allocation, and data loading.
+
+Exports:
+  - Chrome trace (viewable in chrome://tracing)
+  - TensorBoard plugin data (if TB available)
+  - Summary table to console
+
+Usage:
+    profiler = TrainingProfiler(output_dir, n_warmup=3, n_active=5)
+    for batch_idx, batch in enumerate(loader):
+        with profiler.step_context(batch_idx):
+            # ... training step ...
+        if profiler.is_done(batch_idx):
+            break
+    profiler.export()
+"""
+
+import logging
+from pathlib import Path
+
+import torch
+from torch.profiler import ProfilerActivity, profile, schedule, tensorboard_trace_handler
+
+LOGGER = logging.getLogger("caption_test.profiler")
+
+
+class TrainingProfiler:
+    """PyTorch profiler for first N training batches.
+
+    Args:
+        output_dir: Directory for profiler output.
+        n_warmup: Number of warmup steps (not profiled).
+        n_active: Number of steps to actively profile.
+        n_repeat: Number of profiling cycles.
+        record_shapes: Record tensor shapes.
+        profile_memory: Track memory allocation.
+        with_stack: Record Python call stacks.
+    """
+
+    def __init__(
+        self,
+        output_dir: str | Path,
+        n_warmup: int = 3,
+        n_active: int = 5,
+        n_repeat: int = 1,
+        record_shapes: bool = True,
+        profile_memory: bool = True,
+        with_stack: bool = False,
+    ) -> None:
+        self.output_dir = Path(output_dir) / "profiler"
+        self.output_dir.mkdir(parents=True, exist_ok=True)
+        self.n_warmup = n_warmup
+        self.n_active = n_active
+        self.n_repeat = n_repeat
+        self.total_steps = (n_warmup + n_active) * n_repeat
+
+        self._profiler = profile(
+            activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA],
+            schedule=schedule(
+                wait=0,
+                warmup=n_warmup,
+                active=n_active,
+                repeat=n_repeat,
+            ),
+            on_trace_ready=tensorboard_trace_handler(str(self.output_dir)),
+            record_shapes=record_shapes,
+            profile_memory=profile_memory,
+            with_stack=with_stack,
+        )
+        self._started = False
+
+    def start(self) -> None:
+        """Start the profiler."""
+        self._profiler.__enter__()
+        self._started = True
+        LOGGER.info(
+            "Profiler started: %d warmup + %d active steps, output: %s",
+            self.n_warmup, self.n_active, self.output_dir,
+        )
+
+    def step(self) -> None:
+        """Signal end of a profiling step."""
+        if self._started:
+            self._profiler.step()
+
+    def is_done(self, batch_idx: int) -> bool:
+        """Check if profiling is complete."""
+        return batch_idx >= self.total_steps
+
+    def export(self) -> None:
+        """Export profiling results and print summary."""
+        if not self._started:
+            return
+
+        self._profiler.__exit__(None, None, None)
+        self._started = False
+
+        # Print key averages summary.
+        summary = self._profiler.key_averages().table(
+            sort_by="cuda_time_total", row_limit=20,
+        )
+        LOGGER.info("Profiler summary (top 20 by CUDA time):\n%s", summary)
+
+        # Export Chrome trace.
+        trace_path = self.output_dir / "chrome_trace.json"
+        self._profiler.export_chrome_trace(str(trace_path))
+        LOGGER.info("Chrome trace exported: %s", trace_path)
+
+        # Memory summary if available.
+        if torch.cuda.is_available():
+            mem_summary = torch.cuda.memory_summary(abbreviated=True)
+            summary_path = self.output_dir / "memory_summary.txt"
+            summary_path.write_text(mem_summary)
+            LOGGER.info("CUDA memory summary: %s", summary_path)
+
+
+def print_model_summary(model: torch.nn.Module, device: str = "cuda") -> str:
+    """Print model summary using torchinfo (if available).
+
+    Falls back to a simple parameter count if torchinfo is not installed.
+
+    Returns:
+        Summary string.
+    """
+    try:
+        from torchinfo import summary as torchinfo_summary
+        info = torchinfo_summary(
+            model,
+            input_data={
+                "drone_img": torch.randn(1, 3, 256, 256, device=device),
+                "sat_img": torch.randn(1, 3, 256, 256, device=device),
+            },
+            col_names=["input_size", "output_size", "num_params", "trainable"],
+            verbose=0,
+            depth=3,
+        )
+        summary_str = str(info)
+        LOGGER.info("Model summary (torchinfo):\n%s", summary_str)
+        return summary_str
+    except ImportError:
+        LOGGER.info("torchinfo not installed, using basic parameter count")
+    except Exception as e:
+        LOGGER.warning("torchinfo failed (%s), using basic parameter count", e)
+
+    # Fallback: simple param count.
+    lines = []
+    total = 0
+    trainable = 0
+    for name, param in model.named_parameters():
+        total += param.numel()
+        if param.requires_grad:
+            trainable += param.numel()
+            lines.append(f"  [trainable] {name}: {list(param.shape)} ({param.numel():,})")
+
+    summary_str = (
+        f"Total parameters: {total:,}\n"
+        f"Trainable parameters: {trainable:,} ({100*trainable/max(total,1):.2f}%)\n"
+        + "\n".join(lines[:30])
+    )
+    LOGGER.info("Model summary:\n%s", summary_str)
+    return summary_str
--- a/vendor_reference/caption_test/src/training/trackers.py
+++ b/vendor_reference/caption_test/src/training/trackers.py
@@ -0,0 +1,206 @@
+from __future__ import annotations
+
+"""Unified experiment tracking: W&B + TensorBoard + CSV.
+
+Auto-detects available backends. Falls back gracefully if wandb/tensorboard
+are not installed.
+
+Usage:
+    tracker = ExperimentTracker(output_dir, config_dict, use_wandb=True, use_tb=True)
+    tracker.log_train(epoch, {"loss": 0.5, "lr": 1e-4})
+    tracker.log_val(epoch, {"r@1_q2g": 0.3})
+    tracker.log_gradients(epoch, grad_norms_dict)
+    tracker.log_image(epoch, "gradcam/drone", image_tensor)
+    tracker.close()
+"""
+
+import logging
+from pathlib import Path
+from typing import Any
+
+import torch
+
+LOGGER = logging.getLogger("caption_test.trackers")
+
+
+def _try_import_wandb():
+    try:
+        import wandb
+        return wandb
+    except ImportError:
+        return None
+
+
+def _try_import_tb():
+    try:
+        from torch.utils.tensorboard import SummaryWriter
+        return SummaryWriter
+    except ImportError:
+        return None
+
+
+class ExperimentTracker:
+    """Unified tracker dispatching to W&B, TensorBoard, and CSV.
+
+    Args:
+        output_dir: Base output directory.
+        config: Dict of hyperparameters to log.
+        use_wandb: Enable Weights & Biases tracking.
+        use_tb: Enable TensorBoard tracking.
+        wandb_project: W&B project name.
+        wandb_run_name: W&B run name (auto-generated if None).
+        wandb_entity: W&B entity (team/user).
+    """
+
+    def __init__(
+        self,
+        output_dir: str | Path,
+        config: dict[str, Any] | None = None,
+        use_wandb: bool = False,
+        use_tb: bool = True,
+        wandb_project: str = "caption-test-gtauav",
+        wandb_run_name: str | None = None,
+        wandb_entity: str | None = None,
+    ) -> None:
+        self.output_dir = Path(output_dir)
+        self._wandb_run = None
+        self._tb_writer = None
+
+        # W&B init.
+        if use_wandb:
+            wandb = _try_import_wandb()
+            if wandb is not None:
+                self._wandb_run = wandb.init(
+                    project=wandb_project,
+                    name=wandb_run_name,
+                    entity=wandb_entity,
+                    config=config or {},
+                    dir=str(self.output_dir),
+                    reinit=True,
+                )
+                LOGGER.info("W&B initialized: %s", self._wandb_run.url)
+            else:
+                LOGGER.warning("wandb not installed, skipping W&B tracking")
+
+        # TensorBoard init.
+        if use_tb:
+            SummaryWriter = _try_import_tb()
+            if SummaryWriter is not None:
+                tb_dir = self.output_dir / "tb_logs"
+                tb_dir.mkdir(parents=True, exist_ok=True)
+                self._tb_writer = SummaryWriter(log_dir=str(tb_dir))
+                LOGGER.info("TensorBoard initialized: %s", tb_dir)
+            else:
+                LOGGER.warning("tensorboard not installed, skipping TB tracking")
+
+    @property
+    def has_wandb(self) -> bool:
+        return self._wandb_run is not None
+
+    @property
+    def has_tb(self) -> bool:
+        return self._tb_writer is not None
+
+    def log_train(self, epoch: int, metrics: dict[str, float], step: int | None = None) -> None:
+        """Log training metrics for an epoch."""
+        if self._wandb_run is not None:
+            self._wandb_run.log(
+                {f"train/{k}": v for k, v in metrics.items()},
+                step=step or epoch,
+            )
+        if self._tb_writer is not None:
+            for k, v in metrics.items():
+                self._tb_writer.add_scalar(f"train/{k}", v, global_step=step or epoch)
+
+    def log_val(self, epoch: int, metrics: dict[str, float], step: int | None = None) -> None:
+        """Log validation metrics."""
+        if self._wandb_run is not None:
+            self._wandb_run.log(
+                {f"val/{k}": v for k, v in metrics.items()},
+                step=step or epoch,
+            )
+        if self._tb_writer is not None:
+            for k, v in metrics.items():
+                self._tb_writer.add_scalar(f"val/{k}", v, global_step=step or epoch)
+
+    def log_gradients(self, epoch: int, grad_norms: dict[str, float], step: int | None = None) -> None:
+        """Log gradient norms per parameter group."""
+        if self._wandb_run is not None:
+            self._wandb_run.log(
+                {f"gradients/{k}": v for k, v in grad_norms.items()},
+                step=step or epoch,
+            )
+        if self._tb_writer is not None:
+            for k, v in grad_norms.items():
+                self._tb_writer.add_scalar(f"gradients/{k}", v, global_step=step or epoch)
+
+    def log_scalar(self, tag: str, value: float, step: int) -> None:
+        """Log a single scalar."""
+        if self._wandb_run is not None:
+            self._wandb_run.log({tag: value}, step=step)
+        if self._tb_writer is not None:
+            self._tb_writer.add_scalar(tag, value, global_step=step)
+
+    def log_image(self, tag: str, image: Any, step: int, caption: str | None = None) -> None:
+        """Log an image (numpy HWC or torch CHW).
+
+        Args:
+            tag: Image tag/name.
+            image: numpy array [H,W,C] or torch tensor [C,H,W].
+            step: Global step.
+            caption: Optional caption for W&B.
+        """
+        if self._wandb_run is not None:
+            wandb = _try_import_wandb()
+            if isinstance(image, torch.Tensor):
+                image_np = image.detach().cpu().permute(1, 2, 0).numpy()
+            else:
+                image_np = image
+            self._wandb_run.log(
+                {tag: wandb.Image(image_np, caption=caption)},
+                step=step,
+            )
+        if self._tb_writer is not None:
+            if isinstance(image, torch.Tensor):
+                self._tb_writer.add_image(tag, image.detach().cpu(), global_step=step)
+            else:
+                self._tb_writer.add_image(tag, image, global_step=step, dataformats="HWC")
+
+    def log_histogram(self, tag: str, values: torch.Tensor, step: int) -> None:
+        """Log a histogram of values (weights, activations, etc.)."""
+        if self._wandb_run is not None:
+            wandb = _try_import_wandb()
+            self._wandb_run.log(
+                {tag: wandb.Histogram(values.detach().cpu().numpy())},
+                step=step,
+            )
+        if self._tb_writer is not None:
+            self._tb_writer.add_histogram(tag, values.detach().cpu(), global_step=step)
+
+    def log_model_graph(self, model: torch.nn.Module, input_example: Any = None) -> None:
+        """Log model graph to TensorBoard (if available)."""
+        if self._tb_writer is not None and input_example is not None:
+            try:
+                self._tb_writer.add_graph(model, input_example)
+            except Exception as e:
+                LOGGER.warning("Failed to log model graph: %s", e)
+
+    def watch_model(self, model: torch.nn.Module, log_freq: int = 100) -> None:
+        """Enable W&B gradient/weight watching."""
+        if self._wandb_run is not None:
+            wandb = _try_import_wandb()
+            wandb.watch(model, log="all", log_freq=log_freq)
+
+    def log_summary(self, summary: dict[str, Any]) -> None:
+        """Log final summary metrics (best R@1, etc.)."""
+        if self._wandb_run is not None:
+            for k, v in summary.items():
+                self._wandb_run.summary[k] = v
+
+    def close(self) -> None:
+        """Flush and close all backends."""
+        if self._tb_writer is not None:
+            self._tb_writer.flush()
+            self._tb_writer.close()
+        if self._wandb_run is not None:
+            self._wandb_run.finish()
--- a/vendor_reference/caption_test/src/training/train_gtauav.py
+++ b/vendor_reference/caption_test/src/training/train_gtauav.py