# Caption Quality Test on UAV-VisLoc

Validate generated text captions by measuring retrieval R@1 lift on UAV-VisLoc.
Uses GeoRSCLIP ViT-B/32 dual encoder with multi-term InfoNCE.

Full analysis: `2_hypotesis/АНАЛИЗ_caption_quality_test_VisLoc.md`

## Structure

```
caption_test/
├── conf/
│   ├── balanced.gin          # Primary test: λ=(1.0, 0.3, 0.3, 0.1)
│   ├── baseline_no_text.gin  # Reference: λ=(1.0, 0, 0, 0)
│   └── text_heavy.gin        # Stress test: λ=(0.5, 0.5, 0.5, 0.2)
├── scripts/
│   ├── generate_captions.py  # Offline caption generation (template/VLM/hybrid)
│   └── compare_runs.py       # Δ R@1 comparison report builder
├── src/
│   ├── datasets/
│   │   └── visloc_with_captions.py
│   ├── models/
│   │   └── dual_encoder.py   # GeoRSCLIP wrapper, projection heads
│   ├── losses/
│   │   └── multi_infonce.py  # 4-term InfoNCE + curriculum + cosine τ
│   ├── training/
│   │   └── train.py          # Main loop
│   └── eval/
│       └── evaluate.py       # R@K metrics, Δ R@1 helper
└── data/                     # (user-provided) VisLoc pairs + captions
```

## Prerequisites

```
torch>=2.0
open_clip_torch
gin-config
Pillow
numpy
```

GeoRSCLIP ViT-B/32 checkpoint: download `RS5M_ViT-B-32.pt` from
`github.com/om-ai-lab/RS5M` and place under `checkpoints/`.

## Workflow

### 1. Generate captions

```bash
python -m scripts.generate_captions \
    --image_root data/visloc/images \
    --pairs_csv data/visloc/pairs.csv \
    --output data/visloc_train.json \
    --strategy hybrid \
    --vlm_refine_ratio 0.1
```

Replace `_placeholder_vlm_caption` in `scripts/generate_captions.py` with real
Qwen2.5-VL or InternVL2 inference before running on production data.

### 2. Train three variants (in parallel or sequentially)

```bash
# Baseline (no captions)
python -m src.training.train --config conf/baseline_no_text.gin

# Balanced (primary, with captions)
python -m src.training.train --config conf/balanced.gin

# Text-heavy (stress test)
python -m src.training.train --config conf/text_heavy.gin
```

### 3. Evaluate each on test split

```python
from src.eval.evaluate import run_evaluation_from_checkpoint

run_evaluation_from_checkpoint(
    checkpoint_path="out/caption_test/balanced/ckpt_epoch029.pt",
    test_manifest="data/visloc_test.json",
    image_root="data/visloc/images",
    output_path="out/caption_test/balanced/eval_report.json",
)
```

### 4. Compare and get verdict

```bash
python -m scripts.compare_runs \
    --baseline_report out/caption_test/baseline_no_text/eval_report.json \
    --full_report    out/caption_test/balanced/eval_report.json \
    --output         out/caption_test/comparison.md
```

## Decision rule (from `compare_runs.py`)

| Δ R@1 (drone→sat) | Verdict |
|---|---|
| ≥ +3% | ✅ PASS — captions informative, proceed to World-UAV |
| +1% to +3% | ⚠️ MARGINAL — add VLM refinement, re-run |
| 0 to +1% | ❌ WEAK — redesign caption pipeline |
| < 0 | ❌❌ HARMFUL — critical bug |

## Expected runtime (RTX 4090, 24 GB)

| Phase | Time |
|---|---|
| Caption generation (6K pairs, hybrid) | ~1 h |
| Single training run (30 epochs, batch 128) | ~2–3 h |
| Full test (3 variants × 3 seeds = 9 runs) | ~30 h |
| Evaluation + comparison | ~30 min |

## Notes on code style

Follows NADEZHDA code style:
- `from __future__ import annotations` everywhere.
- Type hints on all signatures.
- Google-style docstrings.
- `@gin.configurable` on top-level classes.
- Atomic checkpoint saves (`_atomic_save` helper).
- No emojis in code, English-only code comments.

## Relation to NADEZHDA main pipeline

This is an **isolated experimental track**: completely separate from the main
Student/Teacher training under `code_nadezhda/`. Once captions pass the
Δ R@1 ≥ +3% gate here, the hybrid caption generation strategy is applied to
World-UAV 927K, and those captions feed into E1 Teacher training with
Multi-FiLM conditioning (see `АНАЛИЗ_fusion_для_NADEZHDA.md`).

## Files referenced

- `2_hypotesis/АНАЛИЗ_caption_quality_test_VisLoc.md` — full experimental design
- `2_hypotesis/АНАЛИЗ_text_encoder_для_NADEZHDA.md` — why GeoRSCLIP
- `2_hypotesis/АНАЛИЗ_fusion_для_NADEZHDA.md` — where captions land in Teacher
- `2_hypotesis/ROADMAP_E0_E9_unified.md` — phase E_caption (parallel to E0)