# 1. Резюме (Master)

Full multi-modal fusion в MERIDIAN — **холистический pipeline** для Triple-Teacher (DINOv3-L SAT/Web-LVD/7B, frozen, ~356M+) объединяющий **5 modalities × 2 views = 10 channels** через 10 категорий fusion-парадигм. Master synthesis 4 sub-pair reviews + general fusion review.

```
DINOv3-L backbone (frozen):         ~356M
Per-modality adapters (×5):         ~500K (5 × 100K, light)
Multi-FiLM γ,β projections (×5):    ~50K (5 × ~10K)
KARMMA tokens (5 modalities):       ~100K (5 × 20K)
Θ-Average:                          0 (parameter-free)
─────────────────────────────────────────
Total trainable params:             ~650K (~0.2% backbone)
```

### Master Outcomes

1. **Primary fusion mechanism:** Multi-FiLM-Fusion (F43 SSF TPAMI 2024 anchor + F47 zero-init β) — **<1% params overhead, INT8-compatible**
2. **Secondary mechanism:** ACF Condition Token (F39 CAFuser RA-L 2025) — **-54% params vs separate backbones**
3. **Parallel-arm research:** Fusion-Mamba (F44 TMM 2025) — **+5.9% mAP detection benchmark**
4. **Critical ablations (F88 KARMMA):**
    - Two-token missing-modality strategy
    - Θ-Average FB reduction (-81.45% memory)
    - Modality dropout 50% canonical
5. **Missing modality handling** convergent evidence (4 sources): F45 + F88 + F89 + F90 → **dropout p=0.5 canonical**
6. **Production-ready INT8 stack** confirmed (AUDIT_N6 v2: ~0-2 ms fusion overhead)

### Per-Pair Contribution Summary

|Pair|Modality|Expected R@1 gain|Status|
|:-:|:--|:-:|:--|
|**A**|Segmentation|+2-4% (L_seg aux)|Primary aux|
|**B**|Depth + Normals|+2-4% (geometric)|Primary geometric|
|**C**|CHM|+0.5-1.5% (vegetation scenes)|Optional niche|
|**D**|Text|+0.5-1% (visually-ambiguous)|Secondary|
|**Edges**|Edges|+1-2% (with depth synergy)|Supporting|
|**Combined**|Full 5-modal|**+3-5% R@1 total**|Triple-Teacher|
### Top-5 Critical Reads (Master)

|  #  | Paper                     | Venue                 | Year | Role                                             |
| :-: | :------------------------ | :-------------------- | :--: | :----------------------------------------------- |
|  1  | **F43 SSF (Robust PEFT)** | **IEEE TPAMI**        | 2024 | Multi-FiLM anchor (<1% overhead)                 |
|  2  | **F39 CAFuser**           | **IEEE RA-L**         | 2025 | ACF canonical (-54% params)                      |
|  3  | **F88 KARMMA**            | arXiv                 | 2026 | 3 critical ablations (two-token, Θ-Avg, dropout) |
|  4  | **F44 Fusion-Mamba**      | **IEEE TMM**          | 2025 | Parallel-arm (+5.9% mAP)                         |
|  5  | **F45 Flex-MoE**          | **NeurIPS Spotlight** | 2024 | Missing modality bank                            |
# 2. MERIDIAN Triple-Teacher Architecture

```
INPUT (5 modalities × 2 views = 10 channels):
┌──────────────────────────────────────────────────────────┐
│ ├── RGB sat              ┌── RGB UAV                      │
│ ├── Depth (DepthAny v2)   ├── Depth                       │
│ ├── Edges (Canny/HED)     ├── Edges                       │
│ ├── Segmentation (SAM)    ├── Segmentation                │
│ ├── CHM (Lidar/M11 ML)    ├── CHM                         │
│ └── Text caption (VLM)    └── Text caption                │
└──────────────────────────────────────────────────────────┘
                              ↓
DINOv3-L BACKBONE (frozen, ~356M+):
┌──────────────────────────────────────────────────────────┐
│ ├── DINOv3-L SAT-493M    (satellite-specialized)         │
│ ├── DINOv3-L Web-LVD     (web-scale)                     │
│ └── DINOv3-L ViT-7B      (large general)                 │
└──────────────────────────────────────────────────────────┘
                              ↓
FUSION MECHANISM (10 categories, primary: Multi-FiLM-Fusion):
┌──────────────────────────────────────────────────────────┐
│ Per-modality adapters (F39 CAFuser pattern, light):      │
│   ├── edge_adapter      (light Conv → FiLM)              │
│   ├── depth_adapter     (light Conv → FiLM)              │
│   ├── seg_adapter       (light Conv → FiLM)              │
│   ├── chm_adapter       (light Conv → FiLM)              │
│   └── text_adapter      (CLIP encoder → FiLM)            │
│                                                           │
│ Multi-FiLM-Fusion (F43 SSF pattern):                     │
│   F_fused = F_rgb                                         │
│   F_fused = film_edge(F_fused, F_edge)                    │
│   F_fused = film_depth(F_fused, F_depth)                  │
│   F_fused = film_seg(F_fused, F_seg)                      │
│   F_fused = film_chm(F_fused, F_chm)        ← when avail  │
│   F_fused = film_text(F_fused, F_text)      ← when avail  │
│                                                           │
│ Modality dropout p=0.5 (F88+F45+F89+F90 convergent):     │
│   - Two-token KARMMA для каждой modality                 │
│   - Gradual schedule (F90 sigmoid warmup)                │
│                                                           │
│ Θ-Average FB reduction (F88, -81.45% memory):            │
│   Output Teacher embedding compressed                    │
└──────────────────────────────────────────────────────────┘
                              ↓
OUTPUT: 512-dim Teacher embedding per view (+ optional 64-token queries)
                              ↓
       KD signal (E2-E primary, см. ОБЗОР_KD_detailed_v1)
                              ↓
STUDENT SOFIA v7.6 (edge, ~5M Tiny):
┌──────────────────────────────────────────────────────────┐
│ ├── Input: RGB sat + RGB UAV (always)                    │
│ ├── Backbone (Variant-A/E/Q)                              │
│ ├── Asymmetric Heads (SatHead GGeM + UAVHead CHP)         │
│ └── Optional TextFiLM caption-aware                       │
│                                                           │
│ Latency target: <50 ms Jetson Orin NX INT8              │
└──────────────────────────────────────────────────────────┘
```

### Hybrid Pattern (Multi-FiLM + ACF combined)

**MERIDIAN architecture combines:**

1. **Shared backbone (F39 CAFuser pattern):** DINOv3-L processes RGB primary
2. **Per-modality lightweight adapters** (per F39): edge_adapter, depth_adapter, seg_adapter, chm_adapter, text_adapter
3. **Multi-FiLM modulation per stage (F43 SSF):** Each modality contributes γ⊙F + β
4. **Modality dropout p=0.5** training (F88+F45 convergent)
5. **F88 KARMMA two-token** для missing-modality (each modality)
6. **Θ-Average FB reduction** (F88, -81.45% memory)

### Critical Design Choices

| Decision                                 | Rationale                     | Source                 |
| :--------------------------------------- | :---------------------------- | :--------------------- |
| Shared backbone + adapters               | -54% params vs separate       | **F39** CAFuser        |
| Multi-FiLM modulation                    | <1% overhead PEFT             | **F43** SSF TPAMI      |
| Zero-init β identity при init            | Graceful warmup               | **F47** TacFiLM        |
| Two-token missing-modality               | +43% Epic-Kitchens evidence   | **F88** KARMMA         |
| Θ-Average FB reduction                   | -81.45% memory parameter-free | **F88** KARMMA         |
| Modality dropout p=0.5                   | 4-source convergent           | **F45+F88+F89+F90**    |
| Element-wise gating only                 | INT8 compatible               | **F44** DSSF           |
| Cached Tensors Era                       | No on-device modality compute | **F8** SegEarth-R1     |
| Per-modality adapters light (~100K each) | Param budget                  | **F43** PEFT principle |
### Tier-1 (immediate — E1 Teacher fusion benchmark)

1. **Multi-FiLM-Fusion (F43 + F47)** — primary mechanism
2. **ACF (F39 CAFuser) Condition Token** — secondary for ablation
3. **Fusion-Mamba (F44)** — parallel-arm benchmark
4. **Per-modality light adapters** — F39 pattern (-54% params vs separate)
5. **Modality dropout p=0.5** — canonical (4-source convergent)
6. **Θ-Average FB reduction (F88)** — -81.45% memory, INT8-trivial