Pikaliov/caption-test
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Asymmetric encoders + MONA all 24 blocks + 1024-dim + hard negatives

Browse Source 
Architecture changes:
- Asymmetric DINOv3: WEB (drone) + SAT (satellite) with separate MONA
- MONA on all 24 blocks per encoder (was last 12)
- Remove projection, native 1024-dim retrieval space (was 512)
- Total: 748M params, 17.6M trainable (2.35%)

Hard negative memory bank:
- MoCo-style FIFO queue of 4096 detached gallery embeddings
- Each batch: B in-batch + Q queue negatives in InfoNCE
- Queue updated after each forward pass

Training config:
- batch_size=8, grad_accum=8, effective_batch=64
- eval_every=1 (eval + train recall every epoch)
- Max bs=24 with grad checkpointing on RTX 4090

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-24 08:47:33 +03:00
parent 75a4350d18
commit 04d5307221
5 changed files with 142 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
conf/gtauav_balanced.gin
View File

@@ -1,6 +1,6 @@
# GTA-UAV Balanced: GatedFusion with L1/L2/L3 captions on both branches.
# GTA-UAV Balanced: Asymmetric DINOv3 (WEB+SAT) with L1/L2/L3 captions.
# query = sigma(alpha) * drone + (1-sigma(alpha)) * text -> InfoNCE vs gallery
# 10 epochs, DINOv3 + DGTRS-CLIP, MONA + LoRA adapters.
# 10 epochs, MONA all 24 blocks, 1024-dim retrieval, hard negative bank.
#
# NOTE: TrainConfigGTAUAV is registered by train_gtauav.py before gin parsing.
# InfoNCELoss is registered via import below.
@@ -15,9 +15,9 @@ TrainConfigGTAUAV.learning_rate = 1e-4
TrainConfigGTAUAV.text_lr_factor = 0.1
TrainConfigGTAUAV.weight_decay = 1e-4
TrainConfigGTAUAV.grad_clip = 1.0
TrainConfigGTAUAV.grad_accum_steps = 1
TrainConfigGTAUAV.grad_accum_steps = 8
TrainConfigGTAUAV.use_amp = True
TrainConfigGTAUAV.eval_every = 2
TrainConfigGTAUAV.eval_every = 1
TrainConfigGTAUAV.warmup_epochs = 2
TrainConfigGTAUAV.seed = 42
TrainConfigGTAUAV.device = "cuda"
@@ -25,7 +25,7 @@ TrainConfigGTAUAV.device = "cuda"
# ---- Model ----
TrainConfigGTAUAV.init_gate = 0.7
TrainConfigGTAUAV.baseline_mode = False
TrainConfigGTAUAV.shared_encoder = True
TrainConfigGTAUAV.shared_encoder = False
TrainConfigGTAUAV.gradient_checkpointing = True
# ---- Loss ----
@@ -34,6 +34,7 @@ TrainConfigGTAUAV.label_smoothing = 0.1
TrainConfigGTAUAV.weight_q2g = 0.6
TrainConfigGTAUAV.weight_g2q = 0.4
TrainConfigGTAUAV.learnable_temperature = True
TrainConfigGTAUAV.neg_bank_size = 4096
# ---- Output ----
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text"