Add ML diagnostics tooling (W&B, TensorBoard, Grad-CAM, profiler) and gin configs
- Add unified experiment tracker (W&B + TensorBoard) with graceful fallback - Add gradient norm monitoring per param group (MONA, LoRA, MLP, gates, tau) - Add Grad-CAM visualization for DINOv3 drone/satellite encoders - Add PyTorch Profiler wrapper + torchinfo model summary - Add gin-config support to train_gtauav.py with CLI overrides - Add v3 gin configs: gtauav_balanced, gtauav_baseline, gtauav_text_heavy, gtauav_image_heavy - Generate metric plots every epoch (not just on eval) - Set default epochs to 10 - Update README and CLAUDE.md with new tooling and usage docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
89
README.md
89
README.md
@@ -218,10 +218,13 @@ Gallery: sat_img -> GeoRSCLIP -> gallery
|
||||
|
||||
```
|
||||
caption-test/
|
||||
├── conf/ # Gin configs (v2)
|
||||
│ ├── balanced.gin
|
||||
│ ├── baseline_no_text.gin
|
||||
│ └── text_heavy.gin
|
||||
├── conf/ # Gin configs
|
||||
│ ├── gtauav_balanced.gin # GTA-UAV with text (10 epochs, v3)
|
||||
│ ├── gtauav_baseline.gin # GTA-UAV baseline, no text (v3)
|
||||
│ ├── gtauav_text_heavy.gin # GTA-UAV text-heavy gate=0.3 (v3)
|
||||
│ ├── balanced.gin # UAV-GeoLoc with text (v2)
|
||||
│ ├── baseline_no_text.gin # UAV-GeoLoc baseline (v2)
|
||||
│ └── text_heavy.gin # UAV-GeoLoc text-heavy (v2)
|
||||
├── nn_models/ # Pre-trained checkpoints (v3, gitignored)
|
||||
│ ├── DINO_WEB/ # DINOv3 ViT-L/16 LVD-1689M (.pth)
|
||||
│ ├── DINO_SAT/ # DINOv3 ViT-L/16 SAT-493M (.safetensors)
|
||||
@@ -250,7 +253,12 @@ caption-test/
|
||||
│ │ └── multi_infonce.py # InfoNCE with learnable temperature
|
||||
│ ├── training/
|
||||
│ │ ├── train_gtauav.py # Training loop GTA-UAV (v3)
|
||||
│ │ └── train.py # Training loop UAV-GeoLoc (v2)
|
||||
│ │ ├── train.py # Training loop UAV-GeoLoc (v2)
|
||||
│ │ ├── trackers.py # Unified tracker: W&B + TensorBoard
|
||||
│ │ ├── grad_monitor.py # Gradient norm monitoring per group
|
||||
│ │ ├── gradcam.py # Grad-CAM visualization for DINOv3
|
||||
│ │ ├── profiling.py # PyTorch Profiler + torchinfo summary
|
||||
│ │ └── plot_metrics.py # Seaborn/matplotlib metric plots
|
||||
│ └── eval/
|
||||
│ └── evaluate.py # R@K metrics, Delta R@1
|
||||
└── checkpoints/ # GeoRSCLIP RS5M_ViT-B-32.pt (v2)
|
||||
@@ -268,6 +276,17 @@ regex
|
||||
gin-config
|
||||
Pillow
|
||||
numpy
|
||||
pandas
|
||||
matplotlib
|
||||
seaborn
|
||||
```
|
||||
|
||||
### Optional (for extended diagnostics)
|
||||
|
||||
```
|
||||
wandb # Weights & Biases experiment tracking
|
||||
torchinfo # Model summary tables
|
||||
tensorboard # TensorBoard logging (included with torch)
|
||||
```
|
||||
|
||||
## Workflow (V3 — GTA-UAV)
|
||||
@@ -279,26 +298,52 @@ python -m scripts.make_split --output-dir meta
|
||||
python -m scripts.filter_segmentation --output meta/seg_filter.json
|
||||
```
|
||||
|
||||
### 2. Train baseline (no text)
|
||||
### 2. Train with gin configs (recommended)
|
||||
|
||||
```bash
|
||||
# Baseline (no text, 10 epochs)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_baseline.gin \
|
||||
--filter-meta meta/seg_filter.json
|
||||
|
||||
# With captions (L1/L2/L3, 10 epochs)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json
|
||||
|
||||
# Text-heavy (gate=0.3, 70% text weight)
|
||||
python -m src.training.train_gtauav --config conf/gtauav_text_heavy.gin \
|
||||
--filter-meta meta/seg_filter.json
|
||||
```
|
||||
|
||||
### 3. Train without gin (CLI-only)
|
||||
|
||||
```bash
|
||||
python -m src.training.train_gtauav --baseline --filter-meta meta/seg_filter.json
|
||||
```
|
||||
|
||||
### 3. Train with captions (L1/L2/L3)
|
||||
|
||||
```bash
|
||||
python -m src.training.train_gtauav --filter-meta meta/seg_filter.json
|
||||
```
|
||||
|
||||
### 4. Resume from checkpoint
|
||||
### 4. Enable diagnostics
|
||||
|
||||
```bash
|
||||
# W&B + Grad-CAM + PyTorch Profiler
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json --wandb --gradcam --profile
|
||||
|
||||
# Gin parameter overrides from CLI
|
||||
python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
--filter-meta meta/seg_filter.json \
|
||||
--gin-param 'TrainConfigGTAUAV.batch_size=16' 'TrainConfigGTAUAV.epochs=20'
|
||||
```
|
||||
|
||||
CLI flags (`--wandb`, `--gradcam`, `--profile`, `--epochs`, etc.) take priority over gin config.
|
||||
|
||||
### 5. Resume from checkpoint
|
||||
|
||||
```bash
|
||||
python -m src.training.train_gtauav --resume out/gtauav/with_text/ckpt_epoch004.pt \
|
||||
--filter-meta meta/seg_filter.json
|
||||
```
|
||||
|
||||
### 5. Compare and get verdict
|
||||
### 6. Compare and get verdict
|
||||
|
||||
```bash
|
||||
python -m scripts.compare_runs \
|
||||
@@ -307,6 +352,24 @@ python -m scripts.compare_runs \
|
||||
--output out/gtauav/comparison.md
|
||||
```
|
||||
|
||||
### 7. View TensorBoard
|
||||
|
||||
```bash
|
||||
tensorboard --logdir out/gtauav/with_text/tb_logs
|
||||
```
|
||||
|
||||
## Diagnostics & Visualization
|
||||
|
||||
| Tool | Flag | Output | Description |
|
||||
|------|------|--------|-------------|
|
||||
| **TensorBoard** | `--use-tb` (default on) | `{out}/tb_logs/` | Scalars, histograms, images |
|
||||
| **W&B** | `--wandb` | cloud | Full experiment tracking, Grad-CAM images |
|
||||
| **Grad-CAM** | `--gradcam` | `{out}/gradcam/` | DINOv3 attention heatmaps (drone + satellite) |
|
||||
| **PyTorch Profiler** | `--profile` | `{out}/profiler/` | Chrome trace, CUDA timeline, memory |
|
||||
| **torchinfo** | auto | `{out}/model_summary.txt` | Layer-by-layer parameter table |
|
||||
| **Gradient norms** | `--log-grad-norms` (default on) | TB/W&B | Per-group: MONA, LoRA, MLP, gates, tau |
|
||||
| **CSV + plots** | auto | `{out}/logs/` | train.csv, val.csv, PNG plots every epoch |
|
||||
|
||||
## Decision rule
|
||||
|
||||
| Delta R@1 (drone→satellite) | Verdict |
|
||||
|
||||
Reference in New Issue
Block a user