diff --git a/README.md b/README.md index 1db1faa..9a6b286 100644 --- a/README.md +++ b/README.md @@ -280,12 +280,33 @@ python -m src.training.train_gtauav --config conf/gtauav_balanced.gin \ ### Metrics -| Metric | Formula | Direction | -|--------|---------|-----------| -| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | drone → satellite (primary) | -| **Delta R@1** | R@1(with_text) − R@1(baseline) | higher = text helps | +| Metric | Formula | Direction | Computed on | +|--------|---------|-----------|:-----------:| +| **R@K** (Recall at K) | fraction of queries where correct gallery is in top-K | q→g and g→q | train + val | +| **AP** (Average Precision) | mean of 1/(rank+1) across all queries (MRR) | q→g and g→q | train + val | +| **Loss** (InfoNCE) | symmetric cross-entropy on similarity matrix | — | train + val | +| **Delta R@1** | R@1(with_text) − R@1(baseline) | q→g | val only | -Reported: R@1, R@5, R@10 for both q→g and g→q directions. +R@1, R@5, R@10, AP, and loss are computed on both **train** (subset matching test size, clean +transforms) and **val** (full test set) every epoch. Train vs val comparison enables overfitting +detection: if train R@1/AP rises while val stagnates → overfitting. + +**Output CSVs:** + +| File | Content | Updated | +|------|---------|---------| +| `logs/train.csv` | Epoch-level train loss, temperature, gates, lr | Every epoch | +| `logs/val.csv` | Val R@K, AP, loss, gates | Every eval epoch | +| `logs/train_recall.csv` | Train R@K, AP, loss (subset) | Every eval epoch | +| `logs/train_batches.csv` | Per-batch loss, temperature, gates, lr | Every batch | + +**Plots** (auto-generated in `logs/`): + +| Plot | Panels | +|------|--------| +| `train_metrics.png` | Loss, temperature (τ), gates (σ(α)), learning rate | +| `val_metrics.png` | R@K q→g (train vs val), R@K g→q (train vs val), AP (train vs val) | +| `overview.png` | Train+val loss, val R@1, gates + temperature | ### Optimizer & scheduler @@ -521,7 +542,9 @@ tensorboard --logdir out/gtauav/with_text/tb_logs | **torchinfo** | auto | `{out}/model_summary.txt` | Layer-by-layer parameter table | | **Gradient norms** | `--log-grad-norms` (default on) | TB/W&B | Per-group: MONA, LoRA, MLP, gates, tau | | **CSV (per-batch)** | auto | `{out}/logs/train_batches.csv` | Loss, tau, gates, lr for every batch | -| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch averages + seaborn PNG plots | +| **CSV (per-epoch)** | auto | `{out}/logs/train.csv, val.csv` | Epoch loss averages + seaborn plots | +| **CSV (recall)** | auto | `{out}/logs/train_recall.csv` | Train R@K, AP, loss (subset, clean transforms) | +| **Plots** | auto | `{out}/logs/*.png` | train/val loss, R@K, AP, gates, temperature | ## Decision rule