Pikaliov
f374b0dafc
Standalone Claude Code skill repo extracted from CVGL vault (.claude/skills/generate-hypothesis/). Generates testable If/Then/Because hypotheses for the NADEZHDA / SOFIA research project from the literature library. Contents: - SKILL.md — behaviour spec (6-phase pipeline, output contract) - README.md — human-facing entry: when to use, install, examples - templates/hypothesis_full.md — full template (8 required sections) - templates/hypothesis_compact.md — short draft template - reference/method_taxonomy.md — methods to NADEZHDA components mapping Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5.7 KiB
5.7 KiB
Таксономия методов по компонентам NADEZHDA
Справочник для скилла /generate-hypothesis: какие методы из литературы применимы к каким компонентам системы.
Teacher backbone (DINOv3-L, ~300M frozen, patch 16×16)
| Метод | Источник | Применимость |
|---|---|---|
| DINOv3-L (SAT-493M pretraining, Register Tokens, Gram Anchoring) | F4_BB DINOv3 | Текущий выбор — превосходит DINOv2: +6.4 mIoU dense, satellite-native |
| DINOv2-L self-supervised | Устаревший | Заменён на DINOv3-L |
| GeoRSCLIP (RS-specific VL) | F2_BB | Альтернатива: domain-specific VL pretraining |
| CrossEarth (geospatial FM) | F3_BB | Альтернатива: explicit RS domain |
| Perception Encoder | B13 | Best features at intermediate layers |
Технические особенности DINOv3 vs DINOv2:
- Patch: 16×16 (не 14×14) → 256/16 = 16×16 = 256 токенов (точно)
- Register Tokens: 4 шт. → отфильтровать перед FPN
- Outlier dimensions: ОБЯЗАТЕЛЬНО LayerNorm перед loss-функциями
- Frozen > fine-tuned (by design) → не нужен LoRA
Student backbone (FastViT-T12, ~8.5M, weight-shared)
| Метод | Источник | Применимость |
|---|---|---|
| FastViT-T12 | HYP_Student | Текущий выбор, weight-shared drone/sat |
| MambaVision-T | B12 | Гибрид Mamba+ViT, 4.4 GFLOPs, TensorRT |
| Efficient VMamba | B6 | Atrous selective scan, <50 GFLOPs |
| GroupMamba | B11 | Group-based SSM, ещё легче |
| LEGNet | B16 | 5-8M params, <5 GFLOPs — target-size |
| Channel Group Pooling (CGP) | P1 VimGeo | Замена FC, экономия ~40% params |
| RepViT-M1.1 | Benchmarking FM | 8.2M, 1.7 GFLOPs |
Fusion module (Multi-FiLM-Fusion)
| Метод | Источник | Применимость |
|---|---|---|
| Multi-FiLM (γ·F + β) | F14 WeatherPrompt, SSF TPAMI | Текущий выбор, <0.7% overhead |
| Bottleneck Cross-Attention | GLEAM M3 | Sparse tokens (8/modality) |
| Coupled Mamba | NeurIPS 2024 | Inter-modal SSM state transition, 49% faster |
| Cross-Selective SSM Scan | Sigma WACV 2025 | Mamba-native fusion |
| Mutable Token + Dropout | P50 MMGeo | Graceful degradation при отсутствии модальности |
| MoE (Flex-MoE) | NeurIPS Spotlight | Arbitrary modality combinations |
| Gated Multimodal Unit | Pipeline v1 | Sigmoid gates + weighted sum |
Loss functions (7 losses + GradNorm)
| Loss | Формула | Источник | Роль |
|---|---|---|---|
| L_task | Symmetric InfoNCE (τ learnable, ls=0.1) | P10 Sample4Geo | Основная retrieval |
| L_LUPI | MSE(z_S, sg(z_T_fused)) | R9 Vapnik, R6 MobileGeo | Privileged KD |
| L_feat | MSE(proj(F_S), sg(F_T)) через Conv1×1 | R1, R14 | Intermediate distill |
| L_RKD | d_S(i,j) - d_T(i,j) | ² | |
| L_seg | hard CE + soft KL | Pipeline v2 | Segmentation distill |
| L_CVD_MI | HSIC penalty | disentangle.py | Content-viewpoint separation |
| L_CVD_Recon | Cross-view MSE | disentangle.py | Reconstruction from content only |
| GradNorm | Adaptive λ balancing | Pipeline v2 | Gradient dominance prevention |
| PALW | Progressive attenuate→enhance | P47 DPHR | Альтернатива GradNorm |
| DWBL | Dynamic weighted batch-tuple | P1 VimGeo | Hard negative emphasis |
Distillation strategies
| Стратегия | Источник | Compression | R@1 |
|---|---|---|---|
| MobileGeo hierarchical KD | R6 | 60× | 97.15% (Univ-1652) |
| QDFL frozen + adapters | P62 | <5M adapter | 95.00% |
| LUPI + multi-modal teacher | R9, R3 | Theoretical | — |
| Teacher-Assistant chain | R15 MST-Distill | При capacity gap | — |
| Self-distillation | R4 GeoDistill | Geometry-guided | — |
Edge deployment (Jetson Orin NX)
| Техника | Источник | Эффект |
|---|---|---|
| INT8 TensorRT | R5, R12 | 2-3× speedup, 1-2% R@1 drop |
| k-scaled PTQ для Mamba | R5 | FP16 state transitions + INT8 weights |
| All-adder networks | R12 Q-A²NN | mul→add (экспериментально) |
| Structured pruning | R16 survey | 5% incremental |
| ONNX export | B12 MambaVision | Native TensorRT support |
Data augmentation
| Техника | Источник | Эффект |
|---|---|---|
| CHSG layout simulation | P11 GeoDTR+ | +16-22% cross-domain R@1 |
| Orientation randomization | P16 ConGeo | Предотвращает 16% collapse |
| Weather simulation (10 conditions) | M1 WeatherPrompt | All-weather robustness |
| GPS→DSS hard negatives | P10 Sample4Geo | k=64, K=128, refresh every 4 epochs |
| Modality dropout p=0.3 | P50 MMGeo | LUPI compatibility |
Датасеты
| Датасет | Роль в проекте | Размер | Ключевая метрика |
|---|---|---|---|
| University-1652 | Основной бенчмарк | 142K imgs | D→S R@1 (SOTA: 97.50%) |
| GeoText-1652 | Text modality training | 316K+ texts | Text→Image R@1 |
| GTA-UAV/Game4Loc | Pretraining | 33.7K+14.6K | Zero-shot transfer |
| UAV-VisLoc | Real-world validation | 6,742 drone | Continuous map matching |
| World-UAV | Multi-country test | 927K imgs | Rotation/height robustness |
| CVUSA | Cross-view benchmark | 35K pairs | R@1 (SOTA: 98.97%) |
| CVACT | Cross-domain test | 128K pairs | R@1 (SSM лучше: 81.69%) |
| VIGOR | Hardest benchmark | 90K+105K | Same-area R@1 |
| SUES-200 | Multi-altitude | 24K real | AP across 4 heights |