pikaliov
219bb779eb
Update docs: full architecture with tensor shapes, formulas, optimizer details
...
README: architecture diagram with tensor dimensions, L1/L2/L3 text hierarchy
description, text fusion formula, InfoNCE loss formula with learnable
temperature, metrics table, optimizer/scheduler details with per-group LR,
augmentation table, model parameter summary.
CLAUDE.md: updated to DGTRS-CLIP (official architecture), loss formula,
optimizer/scheduler details, text encoder architecture notes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 18:47:38 +03:00
pikaliov
a47dd6308e
Add architecture diagram to README, update docs for DGTRS text encoder
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 18:36:24 +03:00
pikaliov
905b9867c8
Add 80/20 random split (replaces cross-area 46/54 split)
...
- scripts/make_split.py: merges cross-area train+test (33,708 pairs),
shuffles with seed=42, splits 80/20
- meta/train_80.json (26,966) + meta/test_20.json (6,742)
- After seg filter: 24,891 train / 6,252 test
- Default paths in train_gtauav.py updated to use new split
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 18:19:37 +03:00
pikaliov
3014a5def8
Update docs: training improvements (learnable temp, augmentations, warmup)
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 18:08:06 +03:00
pikaliov
6ad9c4d149
Add GTA-UAV experiment: asymmetric DINOv3 + LRSCLIP text encoder
...
V3 architecture for CVGL caption validation on GTA-UAV-LR dataset:
- AsymmetricEncoder: DINOv3 ViT-L/16 (LVD drone + SAT satellite, frozen)
+ LRSCLIP/DGTRS-CLIP ViT-L-14 text encoder (248 tok, partial unfreeze)
- L1/L2/L3 hierarchical captions from VLM-generated descriptions
- TextFusionMLP (concat 3x768 -> MLP -> 512) + GatedFusion
- Segmentation filter: exclude images with >=90% background+water
- 10.9M trainable / 733M total params, 256x256 input
- coloredlogs + tqdm + emoji for training UX
- Baseline mode (--baseline): image-only, no text encoder loaded
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 17:54:27 +03:00
pikaliov
abb3337f8d
Rewrite: GatedFusion architecture + UAV-GeoLoc dataset
...
Architecture v2:
- Query branch: drone + text -> GatedFusion -> proj -> query_emb
- Gallery branch: satellite -> proj -> gallery_emb
- Single InfoNCE loss (asymmetric 0.6/0.4)
- GatedFusion: learnable gated addition (sigma(alpha)*img + (1-sigma(alpha))*text)
- Baseline mode: gate=1.0 (text ignored)
Dataset:
- UAV-GeoLoc loader with template captions from path metadata
- 27 terrain types with predefined features
- Random positive crop sampling per epoch
Configs: balanced (gate=0.7), baseline (no text), text_heavy (gate=0.3)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-17 17:13:00 +03:00
2ce4017ebd
Initial commit: caption quality test on UAV-VisLoc
...
Self-contained experimental track validating generated text captions
via retrieval R@1 lift on UAV-VisLoc.
Architecture: GeoRSCLIP ViT-B/32 dual encoder, 512-dim shared space.
Loss: 4-term InfoNCE (img-img + sat-cap + drone-cap + cap-cap)
with cosine temperature decay, PALW-like curriculum.
Metric: delta R@1 (with text - without text) >= +3% => PASS.
Gin-configured (balanced / baseline_no_text / text_heavy variants).
Follows NADEZHDA code style.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-04-17 00:04:46 +03:00
