fuse_proj: Initial operational package for 3 researchers (Pavlenko/Blizno/Moroz)

Multimodal fusion research on StripNet+GTA-UAV proxy:
- 3 independent fusion tracks: condition-aware (A), token/bottleneck (B), role-aware (C)
- Shared interfaces, protocol, dataset audit, baseline benchmarks
- Canonical version-chain references to vault (SPEC, ANALYSIS, TRIAGE)
- Personalized task plans and decision tables for each researcher
- 3 generated DOCX task assignment files with milestones and DoD checklist
- Full modality dropout diagnostics and missing-modality robustness requirements
- Data contract, benchmark registry, experiment tracking infrastructure

Operational documents:
- docs/00_project/: MERIDIAN context, protocol, repository reuse guide, experiment specification
- docs/01_tasks/: Master assignment + 3 individual researcher tracks + joint integration
- docs/02_references/: Core literature, version-chain bases, code maps
- docs/03_codebase_guides/: Existing code snapshots from vault
- scripts/: gen_task_plans.js (DOCX generation), placeholder infrastructure
- vendor_reference/: Snapshots of caption_test, depth_edges_annotate, existing SOFIA/SegModel code
- reports/, results/, experiments/: Shared output structure for all 3 researchers

3 DOCX files generated from gen_task_plans.js (Times New Roman 14pt, GOST format):
- План_заданий_Павленко_БВ.docx (Condition-Aware track, fusion API owner)
- План_заданий_Близно_МВ.docx (Token/Bottleneck track, benchmark owner)
- План_заданий_Мороз_ЕС.docx (Role-Aware track, data contract owner)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Pikaliov
2026-06-11 17:16:57 +03:00
commit 2c6a00a4ca
155 changed files with 39765 additions and 0 deletions

View File

@@ -0,0 +1,27 @@
# GTA-UAV Balanced (StripNet backbone): StripNet-small + Conv-MONA in last 2 stages.
# Replaces DINOv3 ViT-L/16 with strip-shaped DWConv hierarchical CNN (~28M params,
# 10× smaller than DINOv3). Output 512-dim → projected to 1024 to match retrieval space.
#
# Trainable:
# - Projection (Linear 512→1024): ~525K
# - Conv-MONA in stages 3 & 4 (2 adapters per Block × 6 blocks total): ~2-3M
# - LoRA on DGTRS-CLIP: 147K
# - TextFusionMLP (shared): ~3.4M
# - GatedFusion gates + tau: 3 scalars
#
# StripNet pretrained on ImageNet-1K (head dropped); state-dict naming follows
# upstream Strip-R-CNN repo (`conv_spatial1/2`).
include 'conf/gtauav_balanced.gin'
# ---- Backbone ----
TrainConfigGTAUAV.backbone = "stripnet"
TrainConfigGTAUAV.stripnet_path = "nn_models/STRIPNET/stripnet_s.pth"
TrainConfigGTAUAV.stripnet_mona_last_n_stages = 2 # Conv-MONA in stages 3 & 4 (deepest)
# ---- Model overrides ----
TrainConfigGTAUAV.shared_encoder = True # StripNet always shared (one CNN for both branches)
TrainConfigGTAUAV.mona_bottleneck = 64 # Conv-MONA bottleneck channels
# ---- Output ----
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text_stripnet"

View File

@@ -0,0 +1,8 @@
# GTA-UAV Baseline (StripNet backbone): no text fusion. Reference R@1 for
# computing Δ R@1 against gtauav_balanced_stripnet.gin.
include 'conf/gtauav_balanced_stripnet.gin'
TrainConfigGTAUAV.baseline_mode = True
TrainConfigGTAUAV.output_dir = "out/gtauav/baseline_stripnet"
TrainConfigGTAUAV.use_gradcam = False