# Caption Quality Test for Cross-View Geo-Localization

Validate whether generated text captions improve retrieval R@1 in cross-view
geo-localization (drone-to-satellite). Uses GeoRSCLIP ViT-B/32 dual encoder
with GatedFusion on the query branch.

## Architecture

```
Query:   drone_img + caption -> GatedFusion -> proj -> query_emb
Gallery: sat_img -> proj -> gallery_emb
Loss:    InfoNCE(query, gallery)
```

Baseline: fusion gate = 1.0 (text ignored).

## Structure

```
caption_test/
├── conf/
│   ├── balanced.gin          # Primary: gate init 0.7 (30% text)
│   ├── baseline_no_text.gin  # Reference: gate = 1.0 (no text)
│   └── text_heavy.gin        # Stress: gate init 0.3 (70% text)
├── scripts/
│   ├── generate_captions.py  # Offline caption generation
│   └── compare_runs.py       # Delta R@1 comparison report
├── src/
│   ├── datasets/
│   │   └── visloc_with_captions.py  # UAV-GeoLoc loader + template captions
│   ├── models/
│   │   └── dual_encoder.py   # GeoRSCLIP + GatedFusion + projection heads
│   ├── losses/
│   │   └── multi_infonce.py  # InfoNCE with cosine temperature
│   ├── training/
│   │   └── train.py          # Main training loop
│   └── eval/
│       └── evaluate.py       # R@K metrics, Delta R@1
└── checkpoints/              # RS5M_ViT-B-32.pt (user-provided)
```

## Prerequisites

```
torch>=2.0
open_clip_torch
gin-config
Pillow
numpy
```

GeoRSCLIP checkpoint: download `RS5M_ViT-B-32.pt` from
`github.com/om-ai-lab/RS5M` and place under `checkpoints/`.

## Workflow

### 1. Train baseline (no text)

```bash
python -m src.training.train --config conf/baseline_no_text.gin
```

### 2. Train with captions

```bash
python -m src.training.train --config conf/balanced.gin
```

### 3. Compare and get verdict

```bash
python -m scripts.compare_runs \
    --baseline_report out/caption_test/baseline_no_text/eval_report.json \
    --full_report out/caption_test/balanced/eval_report.json \
    --output out/caption_test/comparison.md
```

## Decision rule

| Delta R@1 (query->gallery) | Verdict |
|---|---|
| >= +3% | PASS -- captions informative, proceed to production |
| +1% to +3% | MARGINAL -- add VLM refinement, re-run |
| 0 to +1% | WEAK -- redesign caption pipeline |
| < 0 | HARMFUL -- critical bug |

## Expected runtime (RTX 4090, 24 GB)

| Phase | Time |
|---|---|
| Single training run (10 epochs, batch 128, 206K queries) | ~15-30 min |
| Full test (baseline + balanced + text_heavy) | ~1-1.5 h |
| Evaluation | ~2-5 min per run |

## Dataset

UAV-GeoLoc Terrain split (from `/mnt/data1tb/cvgl_datasets/UAV-GeoLoc/`):
- Train: 206,108 queries, 94,709 DB crops (140 scenes)
- Val: 62,368 queries, 26,597 DB crops (40 scenes)
- Test: 33,472 queries, 11,684 DB crops (20 scenes)

Template captions generated automatically from path metadata:
```
"Aerial view at 100m facing northwest over volcanic terrain near KilaueaVolcano.
 Plan-view features: lava flows, crater edges, volcanic rock."
```

## Code style

- `from __future__ import annotations` everywhere
- Type hints on all signatures
- Google-style docstrings
- `@gin.configurable` on top-level classes
- No emojis in code, English-only comments