Dataset preparation script with documentation
- Satellite crop generation (512x512, stride 256, resize 256x256) - Route 09 tile stitching (4 tiles -> 44800x33280) - GPS matching drone->crop via vectorized haversine - Index files in UAV-GeoLoc format (train/test query + DB) - positive.json / semi_positive.json / db_postion.txt per route - Route 07 excluded (satellite too narrow) - Fixed: full gallery in DB files, db_postion.txt format, frame_id keys - Fixed: file handle leaks in image processing loops Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
pikaliov
132
README.md
Normal file
132
README.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# UAV-VisLoc Dataset Preparation
|
||||
|
||||
Prepare UAV-VisLoc dataset for cross-view geo-localization retrieval training.
|
||||
Generates satellite crops, GPS-matched drone-crop pairs, and Index files
|
||||
compatible with UAV-GeoLoc format.
|
||||
|
||||
## Pipeline
|
||||
|
||||
```
|
||||
UAV_VisLoc_dataset/ UAV_VisLoc_processed/
|
||||
├── 01/ ├── 01/
|
||||
│ ├── drone/*.JPG (3976x2652) ---> │ ├── drone/*.JPG (256x256)
|
||||
│ ├── satellite01.tif ---> │ ├── DB/img/crop_X_Y.png (256x256)
|
||||
│ └── 01.csv │ ├── DB/db_postion.txt
|
||||
│ │ ├── positive.json
|
||||
├── ... │ └── semi_positive.json
|
||||
├── 09/ ├── ...
|
||||
│ ├── satellite09_01-01.tif --+ ├── 09/ (stitched from 4 tiles)
|
||||
│ ├── satellite09_01-02.tif +--> │ ├── DB/img/ (~22K crops)
|
||||
│ ├── satellite09_02-01.tif | │ └── ...
|
||||
│ └── satellite09_02-02.tif --+ │
|
||||
├── satellite_ coordinates_range.csv └── Index/
|
||||
├── visloc_train.csv ├── train_query.txt
|
||||
└── visloc_test.csv ├── test_query.txt
|
||||
├── train_db.txt
|
||||
├── test_db.txt
|
||||
└── all_db.txt
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
python scripts/prepare_dataset.py \
|
||||
--src /path/to/UAV_VisLoc_dataset \
|
||||
--dst /path/to/UAV_VisLoc_processed \
|
||||
--crop-size 512 --stride 256 --target-size 256
|
||||
```
|
||||
|
||||
To process specific routes only:
|
||||
```bash
|
||||
python scripts/prepare_dataset.py --src ... --dst ... --routes 01 02 03
|
||||
```
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Resize drone images** -> 256x256 JPEG (quality=95)
|
||||
2. **Stitch satellite tiles** for route 09 (4 tiles -> 44800x33280)
|
||||
3. **Crop satellite maps** -> 512x512 patches, stride 256 (50% overlap), resize -> 256x256 PNG
|
||||
4. **Compute GPS** for each crop center from satellite bbox + grid position
|
||||
5. **Match drone -> crops** via vectorized haversine (positive = closest, semi-positive = +-1 in grid)
|
||||
6. **Write metadata**: positive.json, semi_positive.json, db_postion.txt (per route)
|
||||
7. **Generate Index files**: train_query.txt, test_query.txt, train_db.txt, test_db.txt, all_db.txt
|
||||
|
||||
## Output Format (UAV-GeoLoc compatible)
|
||||
|
||||
### Index files
|
||||
|
||||
**train_query.txt / test_query.txt:**
|
||||
```
|
||||
01/drone/01_0001.JPG 0 01/DB/img/crop_5_18.png 01/DB/img/crop_4_17.png ...
|
||||
```
|
||||
Format: `query_path label positive_crop semi_positive_crops...`
|
||||
|
||||
**train_db.txt / test_db.txt / all_db.txt:**
|
||||
```
|
||||
01/DB/img/crop_0_0.png
|
||||
01/DB/img/crop_0_1.png
|
||||
...
|
||||
```
|
||||
Full gallery (all 74,807 crops), identical for train and test (split is by query).
|
||||
|
||||
### Per-route metadata
|
||||
|
||||
**positive.json:**
|
||||
```json
|
||||
{"0001": ["crop_5_18.png"], "0002": ["crop_5_19.png"], ...}
|
||||
```
|
||||
Keys are frame IDs (without route prefix).
|
||||
|
||||
**semi_positive.json:**
|
||||
```json
|
||||
{"0001": ["crop_4_17.png", "crop_4_18.png", ...], ...}
|
||||
```
|
||||
8 neighbors (+-1 in grid) of the positive crop.
|
||||
|
||||
**db_postion.txt** (tab-separated, matching UAV-GeoLoc spelling):
|
||||
```
|
||||
crop_0_0.png 115.97197337 29.77349180 2.68e-06 -2.68e-06
|
||||
```
|
||||
Columns: name, longitude, latitude, scale_lon (deg/px), scale_lat (deg/px).
|
||||
|
||||
## Dataset Statistics
|
||||
|
||||
| Route | Drone | Crops | Region | Satellite (px) |
|
||||
|-------|-------|-------|--------|----------------|
|
||||
| 01 | 817 | 3,811 | Changjiang | 9774x26762 |
|
||||
| 02 | 1,071 | 5,676 | Changjiang | 11482x34291 |
|
||||
| 03 | 768 | 12,648 | Taizhou | 35092x24308 |
|
||||
| 04 | 738 | 10,281 | Taizhou | 18093x38408 |
|
||||
| 05 | 473 | 805 | Yunnan | 9394x6144 |
|
||||
| 06 | 344 | 1,110 | Zhuxi | 8082x9780 |
|
||||
| 07 | -- | -- | Excluded | 3000x170 (too narrow) |
|
||||
| 08 | 1,033 | 10,416 | Huzhou | 43421x16294 |
|
||||
| 09 | 766 | 22,446 | Huzhou | 44800x33280 (stitched) |
|
||||
| 10 | 144 | 432 | Huailai | 6593x5077 |
|
||||
| 11 | 590 | 7,182 | Shandan | 29592x16582 |
|
||||
| **Total** | **6,744** | **74,807** | | |
|
||||
|
||||
Split: 5,060 train / 1,684 test queries. Gallery: 74,807 crops (shared).
|
||||
|
||||
## GPS Matching Quality
|
||||
|
||||
- Median distance drone -> positive crop: **25.9m**
|
||||
- P99: **45.7m**
|
||||
- Known issue: 6 drones in route 06 (06_0093-06_0098) are outside satellite coverage (~1,091m to nearest crop)
|
||||
|
||||
## Satellite Resolution
|
||||
|
||||
All maps: ~0.30 m/pixel (single Google Earth zoom level).
|
||||
One crop 512x512 covers ~154x154m on the ground.
|
||||
|
||||
## Requirements
|
||||
|
||||
```
|
||||
numpy
|
||||
Pillow
|
||||
```
|
||||
|
||||
## Memory
|
||||
|
||||
Peak RAM usage: ~8.7 GB (route 09 stitching: 4 tiles + merged image).
|
||||
Other routes: 1-3 GB.
|
||||
Reference in New Issue
Block a user