From a47dd6308e55bc90c6dfa9086261ac643f8e7630 Mon Sep 17 00:00:00 2001 From: pikaliov Date: Tue, 21 Apr 2026 18:36:24 +0300 Subject: [PATCH] Add architecture diagram to README, update docs for DGTRS text encoder Co-Authored-By: Claude Opus 4.6 (1M context) --- README.md | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 247dd4e..d1211a8 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,35 @@ geo-localization (drone-to-satellite). Asymmetric architecture with domain-specific image encoders and hierarchical text. ``` -Query: drone_img (DINOv3 LVD) + L1/L2/L3 captions (LRSCLIP) -> GatedFusion -> query -Gallery: sat_img (DINOv3 SAT) -> gallery -Loss: InfoNCE(query, gallery) +┌─────────────────────────── QUERY BRANCH ───────────────────────────┐ +│ │ +│ drone_img ──► DINOv3 ViT-L/16 LVD ──► CLS [1024] │ +│ (frozen, 303M) │ │ +│ proj_drone [1024→512] │ +│ │ │ +│ L1 (overview) ──► DGTRS-CLIP ──► [768] ─┐ │ +│ L2 (full desc) ──► DGTRS-CLIP ──► [768] ─┼─ concat [2304] │ +│ L3 (fingerprint) ──► DGTRS-CLIP ──► [768] ─┘ │ │ +│ (248 tok) MLP [2304→768→512] │ +│ │ │ +│ GatedFusion(img_512, text_512) │ +│ │ │ +│ L2-norm ──► query [512] │ +└─────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────── GALLERY BRANCH ─────────────────────────┐ +│ │ +│ sat_img ──► DINOv3 ViT-L/16 SAT ──► CLS [1024] │ +│ (frozen, 303M) │ │ +│ proj_sat [1024→512] │ +│ │ │ +│ L2-norm ──► gallery [512] │ +└─────────────────────────────────────────────────────────────────────┘ + +LOSS: InfoNCE(query, gallery) — symmetric, learnable temperature + weights: 0.6 × q→g + 0.4 × g→q + +BASELINE: gate = 1.0 (text branch disabled, no DGTRS loaded) ``` **Models:**