Update README: AP metric, train/val eval, CSV/plot inventory

- Document AP (MRR) metric computed on both train and val
- Add output CSVs table (train.csv, val.csv, train_recall.csv, train_batches.csv)
- Add plots table (train_metrics, val_metrics with AP panel, overview)
- Update diagnostics table with recall CSV and plots
- Note overfitting detection via train vs val comparison

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
2026-04-22 08:27:41 +03:00
parent 4a05d05ccd
commit 75a4350d18

View File

@@ -280,12 +280,33 @@ python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \
### Metrics
| Metric | Formula | Direction |
|--------|---------|-----------|
| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | drone → satellite (primary) |
| **Delta R@1** | R@1(with_text) R@1(baseline) | higher = text helps |
| Metric | Formula | Direction | Computed on |
|--------|---------|-----------|:-----------:|
| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | q→g and g→q | train + val |
| **AP** (Average Precision) | mean of 1/(rank+1) across all queries (MRR) | q→g and g→q | train + val |
| **Loss** (InfoNCE) | symmetric cross-entropy on similarity matrix | — | train + val |
| **Delta R@1** | R@1(with_text) R@1(baseline) | q→g | val only |
Reported: R@1, R@5, R@10 for both q→g and g→q directions.
R@1, R@5, R@10, AP, and loss are computed on both **train** (subset matching test size, clean
transforms) and **val** (full test set) every epoch. Train vs val comparison enables overfitting
detection: if train R@1/AP rises while val stagnates → overfitting.
**Output CSVs:**
| File | Content | Updated |
|------|---------|---------|
| `logs/train.csv` | Epoch-level train loss, temperature, gates, lr | Every epoch |
| `logs/val.csv` | Val R@K, AP, loss, gates | Every eval epoch |
| `logs/train_recall.csv` | Train R@K, AP, loss (subset) | Every eval epoch |
| `logs/train_batches.csv` | Per-batch loss, temperature, gates, lr | Every batch |
**Plots** (auto-generated in `logs/`):
| Plot | Panels |
|------|--------|
| `train_metrics.png` | Loss, temperature (τ), gates (σ(α)), learning rate |
| `val_metrics.png` | R@K q→g (train vs val), R@K g→q (train vs val), AP (train vs val) |
| `overview.png` | Train+val loss, val R@1, gates + temperature |
### Optimizer & scheduler
@@ -521,7 +542,9 @@ tensorboard --logdir out/gtauav/with_text/tb_logs
| **torchinfo** | auto | `{out}/model_summary.txt` | Layer-by-layer parameter table |
| **Gradient norms** | `--log-grad-norms` (default on) | TB/W&B | Per-group: MONA, LoRA, MLP, gates, tau |
| **CSV (per-batch)** | auto | `{out}/logs/train_batches.csv` | Loss, tau, gates, lr for every batch |
| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch averages + seaborn PNG plots |
| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch loss averages + seaborn plots |
| **CSV (recall)** | auto | `{out}/logs/train_recall.csv` | Train R@K, AP, loss (subset, clean transforms) |
| **Plots** | auto | `{out}/logs/*.png` | train/val loss, R@K, AP, gates, temperature |
## Decision rule