Add ML diagnostics tooling (W&B, TensorBoard, Grad-CAM, profiler) and gin configs
- Add unified experiment tracker (W&B + TensorBoard) with graceful fallback - Add gradient norm monitoring per param group (MONA, LoRA, MLP, gates, tau) - Add Grad-CAM visualization for DINOv3 drone/satellite encoders - Add PyTorch Profiler wrapper + torchinfo model summary - Add gin-config support to train_gtauav.py with CLI overrides - Add v3 gin configs: gtauav_balanced, gtauav_baseline, gtauav_text_heavy, gtauav_image_heavy - Generate metric plots every epoch (not just on eval) - Set default epochs to 10 - Update README and CLAUDE.md with new tooling and usage docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
43
CLAUDE.md
43
CLAUDE.md
@@ -97,7 +97,16 @@ Eval: Resize(256) + CenterCrop(256) + ImageNet normalization.
|
||||
| `src/models/asymmetric_encoder.py` | DINOv3ViT + TextFusionMLP + AsymmetricEncoder + GatedFusion |
|
||||
| `src/datasets/gtauav_dataset.py` | GTA-UAV-LR loader + L1/L2/L3 caption parsing из VLM JSON |
|
||||
| `src/losses/multi_infonce.py` | InfoNCE с learnable temperature (fp32), clamp [0.01, 0.5] |
|
||||
| `src/training/train_gtauav.py` | Training loop с eval, AMP, per-group LR, warmup, --resume |
|
||||
| `src/training/train_gtauav.py` | Training loop с gin, W&B/TB, AMP, per-group LR, warmup, --resume |
|
||||
| `src/training/trackers.py` | Unified experiment tracker: W&B + TensorBoard + CSV |
|
||||
| `src/training/grad_monitor.py` | Gradient norm monitoring per param group |
|
||||
| `src/training/gradcam.py` | Grad-CAM visualization для DINOv3 encoders |
|
||||
| `src/training/profiling.py` | PyTorch Profiler wrapper + torchinfo model summary |
|
||||
| `src/training/plot_metrics.py` | Seaborn/matplotlib plots (каждую эпоху) |
|
||||
| `conf/gtauav_balanced.gin` | With text, gate=0.7, 10 epochs |
|
||||
| `conf/gtauav_baseline.gin` | No text, gate=1.0 |
|
||||
| `conf/gtauav_text_heavy.gin` | Text-heavy, gate=0.3 |
|
||||
| `conf/gtauav_image_heavy.gin` | Image-heavy, gate=0.9 |
|
||||
| `scripts/make_split.py` | 80/20 random split из всех пар |
|
||||
| `scripts/filter_segmentation.py` | Scan segm masks, output meta JSON (exclude >=90% bg+water) |
|
||||
|
||||
@@ -204,6 +213,14 @@ Meta-файл `meta/seg_filter.json`: исключение изображени
|
||||
- Test: 6,742 → 6,252 after seg filter
|
||||
- Скрипт: `python -m scripts.make_split --ratio 0.8 --seed 42`
|
||||
|
||||
### V3 (GTA-UAV, gin)
|
||||
| Конфиг | Gate init | Описание |
|
||||
|--------|-----------|----------|
|
||||
| `conf/gtauav_balanced.gin` | 0.7 (30% text) | **Primary test** |
|
||||
| `conf/gtauav_baseline.gin` | 1.0 (no text) | Reference baseline |
|
||||
| `conf/gtauav_text_heavy.gin` | 0.3 (70% text) | Stress test |
|
||||
| `conf/gtauav_image_heavy.gin` | 0.9 (10% text) | Image-dominant |
|
||||
|
||||
### V2 (UAV-GeoLoc, gin)
|
||||
| Конфиг | Gate init | Описание |
|
||||
|--------|-----------|----------|
|
||||
@@ -218,17 +235,31 @@ Meta-файл `meta/seg_filter.json`: исключение изображени
|
||||
# 1. Filter segmentation (exclude 90%+ background/water)
|
||||
python -m scripts.filter_segmentation --output meta/seg_filter.json
|
||||
|
||||
# 2. Baseline (no text)
|
||||
python -m src.training.train_gtauav --baseline --filter-meta meta/seg_filter.json
|
||||
# 2. Train with gin config (recommended)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json
|
||||
|
||||
# 3. With captions (L1/L2/L3)
|
||||
python -m src.training.train_gtauav --filter-meta meta/seg_filter.json
|
||||
# 3. Baseline (no text)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_baseline.gin \
|
||||
--filter-meta meta/seg_filter.json
|
||||
|
||||
# 4. Compare
|
||||
# 4. With diagnostics (W&B + Grad-CAM + Profiler)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json --wandb --gradcam --profile
|
||||
|
||||
# 5. CLI overrides (gin params take priority)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json \
|
||||
--gin-param 'TrainConfigGTAUAV.batch_size=16'
|
||||
|
||||
# 6. Compare
|
||||
python -m scripts.compare_runs \
|
||||
--baseline_report out/gtauav/baseline/eval_report.json \
|
||||
--full_report out/gtauav/with_text/eval_report.json \
|
||||
--output out/gtauav/comparison.md
|
||||
|
||||
# 7. TensorBoard
|
||||
tensorboard --logdir out/gtauav/with_text/tb_logs
|
||||
```
|
||||
|
||||
### V2 (UAV-GeoLoc)
|
||||
|
||||
Reference in New Issue
Block a user