fuse_proj: Initial operational package for 3 researchers (Pavlenko/Blizno/Moroz)
Multimodal fusion research on StripNet+GTA-UAV proxy: - 3 independent fusion tracks: condition-aware (A), token/bottleneck (B), role-aware (C) - Shared interfaces, protocol, dataset audit, baseline benchmarks - Canonical version-chain references to vault (SPEC, ANALYSIS, TRIAGE) - Personalized task plans and decision tables for each researcher - 3 generated DOCX task assignment files with milestones and DoD checklist - Full modality dropout diagnostics and missing-modality robustness requirements - Data contract, benchmark registry, experiment tracking infrastructure Operational documents: - docs/00_project/: MERIDIAN context, protocol, repository reuse guide, experiment specification - docs/01_tasks/: Master assignment + 3 individual researcher tracks + joint integration - docs/02_references/: Core literature, version-chain bases, code maps - docs/03_codebase_guides/: Existing code snapshots from vault - scripts/: gen_task_plans.js (DOCX generation), placeholder infrastructure - vendor_reference/: Snapshots of caption_test, depth_edges_annotate, existing SOFIA/SegModel code - reports/, results/, experiments/: Shared output structure for all 3 researchers 3 DOCX files generated from gen_task_plans.js (Times New Roman 14pt, GOST format): - План_заданий_Павленко_БВ.docx (Condition-Aware track, fusion API owner) - План_заданий_Близно_МВ.docx (Token/Bottleneck track, benchmark owner) - План_заданий_Мороз_ЕС.docx (Role-Aware track, data contract owner) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
# GTA-UAV Balanced (StripNet backbone): StripNet-small + Conv-MONA in last 2 stages.
|
||||
# Replaces DINOv3 ViT-L/16 with strip-shaped DWConv hierarchical CNN (~28M params,
|
||||
# 10× smaller than DINOv3). Output 512-dim → projected to 1024 to match retrieval space.
|
||||
#
|
||||
# Trainable:
|
||||
# - Projection (Linear 512→1024): ~525K
|
||||
# - Conv-MONA in stages 3 & 4 (2 adapters per Block × 6 blocks total): ~2-3M
|
||||
# - LoRA on DGTRS-CLIP: 147K
|
||||
# - TextFusionMLP (shared): ~3.4M
|
||||
# - GatedFusion gates + tau: 3 scalars
|
||||
#
|
||||
# StripNet pretrained on ImageNet-1K (head dropped); state-dict naming follows
|
||||
# upstream Strip-R-CNN repo (`conv_spatial1/2`).
|
||||
|
||||
include 'conf/gtauav_balanced.gin'
|
||||
|
||||
# ---- Backbone ----
|
||||
TrainConfigGTAUAV.backbone = "stripnet"
|
||||
TrainConfigGTAUAV.stripnet_path = "nn_models/STRIPNET/stripnet_s.pth"
|
||||
TrainConfigGTAUAV.stripnet_mona_last_n_stages = 2 # Conv-MONA in stages 3 & 4 (deepest)
|
||||
|
||||
# ---- Model overrides ----
|
||||
TrainConfigGTAUAV.shared_encoder = True # StripNet always shared (one CNN for both branches)
|
||||
TrainConfigGTAUAV.mona_bottleneck = 64 # Conv-MONA bottleneck channels
|
||||
|
||||
# ---- Output ----
|
||||
TrainConfigGTAUAV.output_dir = "out/gtauav/with_text_stripnet"
|
||||
@@ -0,0 +1,8 @@
|
||||
# GTA-UAV Baseline (StripNet backbone): no text fusion. Reference R@1 for
|
||||
# computing Δ R@1 against gtauav_balanced_stripnet.gin.
|
||||
|
||||
include 'conf/gtauav_balanced_stripnet.gin'
|
||||
|
||||
TrainConfigGTAUAV.baseline_mode = True
|
||||
TrainConfigGTAUAV.output_dir = "out/gtauav/baseline_stripnet"
|
||||
TrainConfigGTAUAV.use_gradcam = False
|
||||
Reference in New Issue
Block a user