Rewrite: GatedFusion architecture + UAV-GeoLoc dataset

Architecture v2:
- Query branch: drone + text -> GatedFusion -> proj -> query_emb
- Gallery branch: satellite -> proj -> gallery_emb
- Single InfoNCE loss (asymmetric 0.6/0.4)
- GatedFusion: learnable gated addition (sigma(alpha)*img + (1-sigma(alpha))*text)
- Baseline mode: gate=1.0 (text ignored)

Dataset:
- UAV-GeoLoc loader with template captions from path metadata
- 27 terrain types with predefined features
- Random positive crop sampling per epoch

Configs: balanced (gate=0.7), baseline (no text), text_heavy (gate=0.3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-17 17:13:00 +03:00
parent 2ce4017ebd
commit abb3337f8d
12 changed files with 1077 additions and 781 deletions

View File

@@ -1,11 +1,9 @@
# Text-heavy configuration — stress test of caption contribution.
# L = 0.5 * L_img_img + 0.5 * L_sat_cap + 0.5 * L_drone_cap + 0.2 * L_cap_cap
# Text-heavy: gate initialized low (more text weight).
# query = sigma(0.3) * drone + 0.7 * text
include 'balanced.gin'
MultiTermInfoNCE.lambda_ii = 0.5
MultiTermInfoNCE.lambda_sc_max = 0.5
MultiTermInfoNCE.lambda_dc_max = 0.5
MultiTermInfoNCE.lambda_cc_max = 0.2
DualEncoderCaptionTest.init_gate = 0.3
GatedFusion.init_gate = 0.3
TrainConfig.output_dir = "out/caption_test/text_heavy"