Remove projections (1024 native), add satellite text, dual GatedFusion

Architecture changes:
- Removed proj_drone/proj_sat (1024→512): retrieval space is now
  DINOv3 native 1024-dim, no information loss from projection
- TextFusionMLP: 2304→1024→1024 (was 2304→768→512), shared between branches
- Gallery branch now uses satellite captions (L1/L2/L3) via shared TextFusionMLP
- Two separate GatedFusion gates: α_q (query) and α_g (gallery)
- For sat images without captions (~57%): gate passes image features through

Dataset changes:
- GTAUAVDataset now loads satellite captions from caption index
- collate_gtauav_batch includes sat_caption_l1/l2/l3

Training loop:
- Passes satellite captions to model forward
- Logs both gate_q and gate_g values

11.1M trainable / 734M total (1.51%)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-21 19:01:30 +03:00
parent 219bb779eb
commit 0c41c1f017
6 changed files with 172 additions and 120 deletions

View File

@@ -145,7 +145,8 @@ class InfoNCELoss(nn.Module):
weight_b2a=self.weight_g2q,
)
gate = embeddings.get("gate", 1.0)
gate_q = embeddings.get("gate_q", embeddings.get("gate", 1.0))
gate_g = embeddings.get("gate_g", 1.0)
if isinstance(tau, float):
tau_out = torch.tensor(tau, device=loss.device)
@@ -155,5 +156,6 @@ class InfoNCELoss(nn.Module):
return {
"total": loss,
"temperature": tau_out,
"gate": torch.tensor(gate, device=loss.device),
"gate_q": torch.tensor(gate_q, device=loss.device),
"gate_g": torch.tensor(gate_g, device=loss.device),
}