Add SafeTensors consolidation stage for zero-copy tensor loading
Bundle all per-image modalities (depth, edge, chm, segm) into a single .safetensors file for fast training DataLoader reads (~0.1ms zero-copy mmap vs ~5ms for 4x PNG). Adds consolidate stage after main pipeline stages, save_safetensors/cleanup_npy config flags, resume support, and 10 new tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,8 @@ PipelineConfig.stages = ['depth', 'edges', 'segmentation', 'chmv2']
|
||||
PipelineConfig.save_npy = False
|
||||
PipelineConfig.save_vis = True
|
||||
PipelineConfig.save_concat = False
|
||||
PipelineConfig.save_safetensors = True
|
||||
PipelineConfig.cleanup_npy = False
|
||||
PipelineConfig.resume = True
|
||||
PipelineConfig.subset = None
|
||||
# Source filter: 'db' = satellite only, 'query' = drone/UAV only, None = both
|
||||
|
||||
Reference in New Issue
Block a user