Update docs: MONA/LoRA architecture, LaTeX formulas, param summary
README: LaTeX math formulas for text fusion, gated fusion, MONA adapter, LoRA, and InfoNCE loss. Added adaptation methods table (MONA + LoRA). Updated model summary to 17.6M/748M (2.35%). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
13
CLAUDE.md
13
CLAUDE.md
@@ -50,14 +50,15 @@ BASELINE: σ(α_q)=σ(α_g)=1.0, text disabled, DGTRS not loaded
|
||||
- Transformer: sequence-first (LND), nn.MultiheadAttention, 12 layers
|
||||
- Tokenizer: BPE SimpleTokenizer (248 tokens, vocab 49408)
|
||||
|
||||
### Trainable parameters: 11.1M из 734M (1.51%)
|
||||
- TextFusionMLP (shared): Linear(2304,1024)+GELU+Linear(1024,1024) = ~3.5M
|
||||
- gate α_q: 1 scalar (query branch)
|
||||
- gate α_g: 1 scalar (gallery branch)
|
||||
### Trainable parameters: 17.6M из 748M (2.35%)
|
||||
- **MONA adapters** (2×DINOv3): 14.0M (2 per block × 24 × 2 encoders, bottleneck=64)
|
||||
- **LoRA** (DGTRS-CLIP): 147K (Q+V, rank=4, 12 blocks)
|
||||
- TextFusionMLP (shared): Linear(2304,1024)+GELU+Linear(1024,1024) = ~3.4M
|
||||
- gate α_q + α_g: 2 scalars
|
||||
- logit_scale: 1 scalar (learnable temperature)
|
||||
- DGTRS partial unfreeze (last resblock + ln_final + text_projection): ~7.6M
|
||||
- DINOv3 x2 (303M each): frozen
|
||||
- DINOv3 x2 + DGTRS: frozen backbone weights
|
||||
- **Без projection layers** — retrieval space = DINOv3 native 1024-dim
|
||||
- **AMP:** frozen layers fp16, adapters + loss fp32
|
||||
|
||||
### Optimizer & Scheduler
|
||||
- **AdamW** с per-group LR: projections lr=1e-4, text encoder lr=1e-5
|
||||
|
||||
Reference in New Issue
Block a user