Pikaliov
f374b0dafc
Standalone Claude Code skill repo extracted from CVGL vault (.claude/skills/generate-hypothesis/). Generates testable If/Then/Because hypotheses for the NADEZHDA / SOFIA research project from the literature library. Contents: - SKILL.md — behaviour spec (6-phase pipeline, output contract) - README.md — human-facing entry: when to use, install, examples - templates/hypothesis_full.md — full template (8 required sections) - templates/hypothesis_compact.md — short draft template - reference/method_taxonomy.md — methods to NADEZHDA components mapping Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
103 lines
5.7 KiB
Markdown
103 lines
5.7 KiB
Markdown
# Таксономия методов по компонентам NADEZHDA
|
||
|
||
Справочник для скилла `/generate-hypothesis`: какие методы из литературы применимы к каким компонентам системы.
|
||
|
||
## Teacher backbone (DINOv3-L, ~300M frozen, patch 16×16)
|
||
|
||
| Метод | Источник | Применимость |
|
||
|:------|:---------|:-------------|
|
||
| **DINOv3-L** (SAT-493M pretraining, Register Tokens, Gram Anchoring) | F4_BB DINOv3 | **Текущий выбор** — превосходит DINOv2: +6.4 mIoU dense, satellite-native |
|
||
| DINOv2-L self-supervised | Устаревший | Заменён на DINOv3-L |
|
||
| GeoRSCLIP (RS-specific VL) | F2_BB | Альтернатива: domain-specific VL pretraining |
|
||
| CrossEarth (geospatial FM) | F3_BB | Альтернатива: explicit RS domain |
|
||
| Perception Encoder | B13 | Best features at intermediate layers |
|
||
|
||
**Технические особенности DINOv3 vs DINOv2:**
|
||
- Patch: 16×16 (не 14×14) → 256/16 = 16×16 = 256 токенов (точно)
|
||
- Register Tokens: 4 шт. → отфильтровать перед FPN
|
||
- Outlier dimensions: ОБЯЗАТЕЛЬНО LayerNorm перед loss-функциями
|
||
- Frozen > fine-tuned (by design) → не нужен LoRA
|
||
|
||
## Student backbone (FastViT-T12, ~8.5M, weight-shared)
|
||
|
||
| Метод | Источник | Применимость |
|
||
|:------|:---------|:-------------|
|
||
| FastViT-T12 | HYP_Student | Текущий выбор, weight-shared drone/sat |
|
||
| MambaVision-T | B12 | Гибрид Mamba+ViT, 4.4 GFLOPs, TensorRT |
|
||
| Efficient VMamba | B6 | Atrous selective scan, <50 GFLOPs |
|
||
| GroupMamba | B11 | Group-based SSM, ещё легче |
|
||
| LEGNet | B16 | 5-8M params, <5 GFLOPs — target-size |
|
||
| Channel Group Pooling (CGP) | P1 VimGeo | Замена FC, экономия ~40% params |
|
||
| RepViT-M1.1 | Benchmarking FM | 8.2M, 1.7 GFLOPs |
|
||
|
||
## Fusion module (Multi-FiLM-Fusion)
|
||
|
||
| Метод | Источник | Применимость |
|
||
|:------|:---------|:-------------|
|
||
| Multi-FiLM (γ·F + β) | F14 WeatherPrompt, SSF TPAMI | Текущий выбор, <0.7% overhead |
|
||
| Bottleneck Cross-Attention | GLEAM M3 | Sparse tokens (8/modality) |
|
||
| Coupled Mamba | NeurIPS 2024 | Inter-modal SSM state transition, 49% faster |
|
||
| Cross-Selective SSM Scan | Sigma WACV 2025 | Mamba-native fusion |
|
||
| Mutable Token + Dropout | P50 MMGeo | Graceful degradation при отсутствии модальности |
|
||
| MoE (Flex-MoE) | NeurIPS Spotlight | Arbitrary modality combinations |
|
||
| Gated Multimodal Unit | Pipeline v1 | Sigmoid gates + weighted sum |
|
||
|
||
## Loss functions (7 losses + GradNorm)
|
||
|
||
| Loss | Формула | Источник | Роль |
|
||
|:-----|:--------|:---------|:-----|
|
||
| L_task | Symmetric InfoNCE (τ learnable, ls=0.1) | P10 Sample4Geo | Основная retrieval |
|
||
| L_LUPI | MSE(z_S, sg(z_T_fused)) | R9 Vapnik, R6 MobileGeo | Privileged KD |
|
||
| L_feat | MSE(proj(F_S), sg(F_T)) через Conv1×1 | R1, R14 | Intermediate distill |
|
||
| L_RKD | |d_S(i,j) - d_T(i,j)|² | R1 | Relational KD |
|
||
| L_seg | hard CE + soft KL | Pipeline v2 | Segmentation distill |
|
||
| L_CVD_MI | HSIC penalty | disentangle.py | Content-viewpoint separation |
|
||
| L_CVD_Recon | Cross-view MSE | disentangle.py | Reconstruction from content only |
|
||
| GradNorm | Adaptive λ balancing | Pipeline v2 | Gradient dominance prevention |
|
||
| PALW | Progressive attenuate→enhance | P47 DPHR | Альтернатива GradNorm |
|
||
| DWBL | Dynamic weighted batch-tuple | P1 VimGeo | Hard negative emphasis |
|
||
|
||
## Distillation strategies
|
||
|
||
| Стратегия | Источник | Compression | R@1 |
|
||
|:----------|:---------|:-----------|:----|
|
||
| MobileGeo hierarchical KD | R6 | 60× | 97.15% (Univ-1652) |
|
||
| QDFL frozen + adapters | P62 | <5M adapter | 95.00% |
|
||
| LUPI + multi-modal teacher | R9, R3 | Theoretical | — |
|
||
| Teacher-Assistant chain | R15 MST-Distill | При capacity gap | — |
|
||
| Self-distillation | R4 GeoDistill | Geometry-guided | — |
|
||
|
||
## Edge deployment (Jetson Orin NX)
|
||
|
||
| Техника | Источник | Эффект |
|
||
|:--------|:---------|:-------|
|
||
| INT8 TensorRT | R5, R12 | 2-3× speedup, 1-2% R@1 drop |
|
||
| k-scaled PTQ для Mamba | R5 | FP16 state transitions + INT8 weights |
|
||
| All-adder networks | R12 Q-A²NN | mul→add (экспериментально) |
|
||
| Structured pruning | R16 survey | 5% incremental |
|
||
| ONNX export | B12 MambaVision | Native TensorRT support |
|
||
|
||
## Data augmentation
|
||
|
||
| Техника | Источник | Эффект |
|
||
|:--------|:---------|:-------|
|
||
| CHSG layout simulation | P11 GeoDTR+ | +16-22% cross-domain R@1 |
|
||
| Orientation randomization | P16 ConGeo | Предотвращает 16% collapse |
|
||
| Weather simulation (10 conditions) | M1 WeatherPrompt | All-weather robustness |
|
||
| GPS→DSS hard negatives | P10 Sample4Geo | k=64, K=128, refresh every 4 epochs |
|
||
| Modality dropout p=0.3 | P50 MMGeo | LUPI compatibility |
|
||
|
||
## Датасеты
|
||
|
||
| Датасет | Роль в проекте | Размер | Ключевая метрика |
|
||
|:--------|:---------------|:-------|:-----------------|
|
||
| University-1652 | Основной бенчмарк | 142K imgs | D→S R@1 (SOTA: 97.50%) |
|
||
| GeoText-1652 | Text modality training | 316K+ texts | Text→Image R@1 |
|
||
| GTA-UAV/Game4Loc | Pretraining | 33.7K+14.6K | Zero-shot transfer |
|
||
| UAV-VisLoc | Real-world validation | 6,742 drone | Continuous map matching |
|
||
| World-UAV | Multi-country test | 927K imgs | Rotation/height robustness |
|
||
| CVUSA | Cross-view benchmark | 35K pairs | R@1 (SOTA: 98.97%) |
|
||
| CVACT | Cross-domain test | 128K pairs | R@1 (SSM лучше: 81.69%) |
|
||
| VIGOR | Hardest benchmark | 90K+105K | Same-area R@1 |
|
||
| SUES-200 | Multi-altitude | 24K real | AP across 4 heights |
|