Update README: AP metric, train/val eval, CSV/plot inventory
- Document AP (MRR) metric computed on both train and val - Add output CSVs table (train.csv, val.csv, train_recall.csv, train_batches.csv) - Add plots table (train_metrics, val_metrics with AP panel, overview) - Update diagnostics table with recall CSV and plots - Note overfitting detection via train vs val comparison Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
35
README.md
35
README.md
@@ -280,12 +280,33 @@ python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
|
||||
|
||||
### Metrics
|
||||
|
||||
| Metric | Formula | Direction |
|
||||
|--------|---------|-----------|
|
||||
| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | drone → satellite (primary) |
|
||||
| **Delta R@1** | R@1(with_text) − R@1(baseline) | higher = text helps |
|
||||
| Metric | Formula | Direction | Computed on |
|
||||
|--------|---------|-----------|:-----------:|
|
||||
| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | q→g and g→q | train + val |
|
||||
| **AP** (Average Precision) | mean of 1/(rank+1) across all queries (MRR) | q→g and g→q | train + val |
|
||||
| **Loss** (InfoNCE) | symmetric cross-entropy on similarity matrix | — | train + val |
|
||||
| **Delta R@1** | R@1(with_text) − R@1(baseline) | q→g | val only |
|
||||
|
||||
Reported: R@1, R@5, R@10 for both q→g and g→q directions.
|
||||
R@1, R@5, R@10, AP, and loss are computed on both **train** (subset matching test size, clean
|
||||
transforms) and **val** (full test set) every epoch. Train vs val comparison enables overfitting
|
||||
detection: if train R@1/AP rises while val stagnates → overfitting.
|
||||
|
||||
**Output CSVs:**
|
||||
|
||||
| File | Content | Updated |
|
||||
|------|---------|---------|
|
||||
| `logs/train.csv` | Epoch-level train loss, temperature, gates, lr | Every epoch |
|
||||
| `logs/val.csv` | Val R@K, AP, loss, gates | Every eval epoch |
|
||||
| `logs/train_recall.csv` | Train R@K, AP, loss (subset) | Every eval epoch |
|
||||
| `logs/train_batches.csv` | Per-batch loss, temperature, gates, lr | Every batch |
|
||||
|
||||
**Plots** (auto-generated in `logs/`):
|
||||
|
||||
| Plot | Panels |
|
||||
|------|--------|
|
||||
| `train_metrics.png` | Loss, temperature (τ), gates (σ(α)), learning rate |
|
||||
| `val_metrics.png` | R@K q→g (train vs val), R@K g→q (train vs val), AP (train vs val) |
|
||||
| `overview.png` | Train+val loss, val R@1, gates + temperature |
|
||||
|
||||
### Optimizer & scheduler
|
||||
|
||||
@@ -521,7 +542,9 @@ tensorboard --logdir out/gtauav/with_text/tb_logs
|
||||
| **torchinfo** | auto | `{out}/model_summary.txt` | Layer-by-layer parameter table |
|
||||
| **Gradient norms** | `--log-grad-norms` (default on) | TB/W&B | Per-group: MONA, LoRA, MLP, gates, tau |
|
||||
| **CSV (per-batch)** | auto | `{out}/logs/train_batches.csv` | Loss, tau, gates, lr for every batch |
|
||||
| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch averages + seaborn PNG plots |
|
||||
| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch loss averages + seaborn plots |
|
||||
| **CSV (recall)** | auto | `{out}/logs/train_recall.csv` | Train R@K, AP, loss (subset, clean transforms) |
|
||||
| **Plots** | auto | `{out}/logs/*.png` | train/val loss, R@K, AP, gates, temperature |
|
||||
|
||||
## Decision rule
|
||||
|
||||
|
||||
Reference in New Issue
Block a user