Update docs: MONA/LoRA architecture, LaTeX formulas, param summary

README: LaTeX math formulas for text fusion, gated fusion, MONA adapter,
LoRA, and InfoNCE loss. Added adaptation methods table (MONA + LoRA).
Updated model summary to 17.6M/748M (2.35%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-21 19:26:01 +03:00
parent a39f8a9655
commit 082f82f138
2 changed files with 61 additions and 46 deletions

View File

@@ -50,14 +50,15 @@ BASELINE: σ(α_q)=σ(α_g)=1.0, text disabled, DGTRS not loaded
- Transformer: sequence-first (LND), nn.MultiheadAttention, 12 layers
- Tokenizer: BPE SimpleTokenizer (248 tokens, vocab 49408)
### Trainable parameters: 11.1M из 734M (1.51%)
- TextFusionMLP (shared): Linear(2304,1024)+GELU+Linear(1024,1024) = ~3.5M
- gate α_q: 1 scalar (query branch)
- gate α_g: 1 scalar (gallery branch)
### Trainable parameters: 17.6M из 748M (2.35%)
- **MONA adapters** (2×DINOv3): 14.0M (2 per block × 24 × 2 encoders, bottleneck=64)
- **LoRA** (DGTRS-CLIP): 147K (Q+V, rank=4, 12 blocks)
- TextFusionMLP (shared): Linear(2304,1024)+GELU+Linear(1024,1024) = ~3.4M
- gate α_q + α_g: 2 scalars
- logit_scale: 1 scalar (learnable temperature)
- DGTRS partial unfreeze (last resblock + ln_final + text_projection): ~7.6M
- DINOv3 x2 (303M each): frozen
- DINOv3 x2 + DGTRS: frozen backbone weights
- **Без projection layers** — retrieval space = DINOv3 native 1024-dim
- **AMP:** frozen layers fp16, adapters + loss fp32
### Optimizer & Scheduler
- **AdamW** с per-group LR: projections lr=1e-4, text encoder lr=1e-5