Pikaliov/caption-test
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite: GatedFusion architecture + UAV-GeoLoc dataset

Browse Source 
Architecture v2:
- Query branch: drone + text -> GatedFusion -> proj -> query_emb
- Gallery branch: satellite -> proj -> gallery_emb
- Single InfoNCE loss (asymmetric 0.6/0.4)
- GatedFusion: learnable gated addition (sigma(alpha)*img + (1-sigma(alpha))*text)
- Baseline mode: gate=1.0 (text ignored)

Dataset:
- UAV-GeoLoc loader with template captions from path metadata
- 27 terrain types with predefined features
- Random positive crop sampling per epoch

Configs: balanced (gate=0.7), baseline (no text), text_heavy (gate=0.3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-17 17:13:00 +03:00
parent 2ce4017ebd
commit abb3337f8d
12 changed files with 1077 additions and 781 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
conf/baseline_no_text.gin
View File

@@ -1,11 +1,10 @@
# Baseline: image-image only, no captions. Reference R@1 for delta computation.
# L = 1.0 * L_img_img
# Baseline: no text fusion (gate forced to 1.0).
# query = drone_only -> InfoNCE vs satellite
# Reference R@1 for delta computation.
include 'balanced.gin'
# Disable all caption loss terms.
MultiTermInfoNCE.lambda_sc_max = 0.0
MultiTermInfoNCE.lambda_dc_max = 0.0
MultiTermInfoNCE.lambda_cc_max = 0.0
DualEncoderCaptionTest.baseline_mode = True
GatedFusion.baseline_mode = True
TrainConfig.output_dir = "out/caption_test/baseline_no_text"