Add ML diagnostics tooling (W&B, TensorBoard, Grad-CAM, profiler) and gin configs

- Add unified experiment tracker (W&B + TensorBoard) with graceful fallback
- Add gradient norm monitoring per param group (MONA, LoRA, MLP, gates, tau)
- Add Grad-CAM visualization for DINOv3 drone/satellite encoders
- Add PyTorch Profiler wrapper + torchinfo model summary
- Add gin-config support to train_gtauav.py with CLI overrides
- Add v3 gin configs: gtauav_balanced, gtauav_baseline, gtauav_text_heavy, gtauav_image_heavy
- Generate metric plots every epoch (not just on eval)
- Set default epochs to 10
- Update README and CLAUDE.md with new tooling and usage docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-21 20:30:50 +03:00
parent 83ce04150d
commit 29a09349e7
11 changed files with 1098 additions and 57 deletions

View File

@@ -97,7 +97,16 @@ Eval: Resize(256) + CenterCrop(256) + ImageNet normalization.
| `src/models/asymmetric_encoder.py` | DINOv3ViT + TextFusionMLP + AsymmetricEncoder + GatedFusion |
| `src/datasets/gtauav_dataset.py` | GTA-UAV-LR loader + L1/L2/L3 caption parsing из VLM JSON |
| `src/losses/multi_infonce.py` | InfoNCE с learnable temperature (fp32), clamp [0.01, 0.5] |
| `src/training/train_gtauav.py` | Training loop с eval, AMP, per-group LR, warmup, --resume |
| `src/training/train_gtauav.py` | Training loop с gin, W&B/TB, AMP, per-group LR, warmup, --resume |
| `src/training/trackers.py` | Unified experiment tracker: W&B + TensorBoard + CSV |
| `src/training/grad_monitor.py` | Gradient norm monitoring per param group |
| `src/training/gradcam.py` | Grad-CAM visualization для DINOv3 encoders |
| `src/training/profiling.py` | PyTorch Profiler wrapper + torchinfo model summary |
| `src/training/plot_metrics.py` | Seaborn/matplotlib plots (каждую эпоху) |
| `conf/gtauav_balanced.gin` | With text, gate=0.7, 10 epochs |
| `conf/gtauav_baseline.gin` | No text, gate=1.0 |
| `conf/gtauav_text_heavy.gin` | Text-heavy, gate=0.3 |
| `conf/gtauav_image_heavy.gin` | Image-heavy, gate=0.9 |
| `scripts/make_split.py` | 80/20 random split из всех пар |
| `scripts/filter_segmentation.py` | Scan segm masks, output meta JSON (exclude >=90% bg+water) |
@@ -204,6 +213,14 @@ Meta-файл `meta/seg_filter.json`: исключение изображени
- Test: 6,742 → 6,252 after seg filter
- Скрипт: `python -m scripts.make_split --ratio 0.8 --seed 42`
### V3 (GTA-UAV, gin)
| Конфиг | Gate init | Описание |
|--------|-----------|----------|
| `conf/gtauav_balanced.gin` | 0.7 (30% text) | **Primary test** |
| `conf/gtauav_baseline.gin` | 1.0 (no text) | Reference baseline |
| `conf/gtauav_text_heavy.gin` | 0.3 (70% text) | Stress test |
| `conf/gtauav_image_heavy.gin` | 0.9 (10% text) | Image-dominant |
### V2 (UAV-GeoLoc, gin)
| Конфиг | Gate init | Описание |
|--------|-----------|----------|
@@ -218,17 +235,31 @@ Meta-файл `meta/seg_filter.json`: исключение изображени
# 1. Filter segmentation (exclude 90%+ background/water)
python -m scripts.filter_segmentation --output meta/seg_filter.json
# 2. Baseline (no text)
python -m src.training.train_gtauav --baseline --filter-meta meta/seg_filter.json
# 2. Train with gin config (recommended)
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
--filter-meta meta/seg_filter.json
# 3. With captions (L1/L2/L3)
python -m src.training.train_gtauav --filter-meta meta/seg_filter.json
# 3. Baseline (no text)
python -m src.training.train_gtauav --config conf/gtauav_baseline.gin \
--filter-meta meta/seg_filter.json
# 4. Compare
# 4. With diagnostics (W&B + Grad-CAM + Profiler)
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
--filter-meta meta/seg_filter.json --wandb --gradcam --profile
# 5. CLI overrides (gin params take priority)
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
--filter-meta meta/seg_filter.json \
--gin-param 'TrainConfigGTAUAV.batch_size=16'
# 6. Compare
python -m scripts.compare_runs \
--baseline_report out/gtauav/baseline/eval_report.json \
--full_report out/gtauav/with_text/eval_report.json \
--output out/gtauav/comparison.md
# 7. TensorBoard
tensorboard --logdir out/gtauav/with_text/tb_logs
```
### V2 (UAV-GeoLoc)