- DINOv3: checkpoint each of 24 transformer blocks (recompute on backward)
- DGTRS-CLIP: checkpoint each of 12 transformer blocks
- Enables batch_size=24 on RTX 4090 (was 8 without checkpointing)
- Peak VRAM: 20.3 GB at bs=24 (was OOM at bs=16 before)
- ~20-30% slower per step, but 3x more in-batch negatives (23 vs 7)
- Enabled by default (gradient_checkpointing=True in config)
- Update README with VRAM benchmarks and checkpointing docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Single DINOv3 WEB for both drone and satellite branches (shared_encoder=True default)
- One set of MONA adapters instead of two: 7M trainable vs 14M
- Total params: 438M (was 748M), trainable: 10.6M (was 17.6M)
- Asymmetric mode still available via shared_encoder=False
- Add gradient accumulation (grad_accum_steps, --grad-accum CLI flag)
- Update model summary in README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New config field grad_accum_steps (default=1, no change in behavior)
- Loss scaled by 1/accum, optimizer step every N micro-batches
- Scheduler counts optimizer steps (not micro-batches)
- CLI flag --grad-accum for override
- Document gradient accumulation and in-batch negatives in README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use gin.configurable(module=...) to prevent __main__ vs module name clash
- Remove `import src.training.train_gtauav` from gin files (already loaded)
- Use short selector names (TrainConfigGTAUAV) in all gin configs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add unified experiment tracker (W&B + TensorBoard) with graceful fallback
- Add gradient norm monitoring per param group (MONA, LoRA, MLP, gates, tau)
- Add Grad-CAM visualization for DINOv3 drone/satellite encoders
- Add PyTorch Profiler wrapper + torchinfo model summary
- Add gin-config support to train_gtauav.py with CLI overrides
- Add v3 gin configs: gtauav_balanced, gtauav_baseline, gtauav_text_heavy, gtauav_image_heavy
- Generate metric plots every epoch (not just on eval)
- Set default epochs to 10
- Update README and CLAUDE.md with new tooling and usage docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
