Initial commit — scientific-viz skill
Claude Code skill for publication-quality scientific visualisations: chart, architecture, flowchart, comparison, pipeline. Bundles matplotlib publication defaults (300 DPI, serif font, colorblind palette), NADEZHDA project palette, Mermaid conventions, and ready-to-use templates. Contents: - SKILL.md — behaviour spec (5 types, generation pipeline) - README.md — human-facing entry: when, install, examples - reference/chart_patterns.md — Pareto/bar/radar/heatmap snippets - templates/architecture_nadezhda.md — Mermaid templates for Teacher-Student / LUPI / fusion Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
58
.gitignore
vendored
Normal file
58
.gitignore
vendored
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
# Python build artifacts
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*$py.class
|
||||||
|
*.egg-info/
|
||||||
|
*.egg
|
||||||
|
.eggs/
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
|
||||||
|
# Virtual environments
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
env/
|
||||||
|
ENV/
|
||||||
|
.python-version
|
||||||
|
|
||||||
|
# Coverage / testing
|
||||||
|
.coverage
|
||||||
|
.coverage.*
|
||||||
|
htmlcov/
|
||||||
|
.tox/
|
||||||
|
.nox/
|
||||||
|
.pytest_cache/
|
||||||
|
.mypy_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
.dmypy.json
|
||||||
|
|
||||||
|
# IDE / OS
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
!.vscode/settings.json
|
||||||
|
!.vscode/launch.json
|
||||||
|
!.vscode/extensions.json
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
desktop.ini
|
||||||
|
|
||||||
|
# Jupyter
|
||||||
|
.ipynb_checkpoints/
|
||||||
|
|
||||||
|
# Secrets — never commit
|
||||||
|
.env
|
||||||
|
.env.local
|
||||||
|
*.key
|
||||||
|
*.pem
|
||||||
|
credentials.json
|
||||||
|
|
||||||
|
# Temp
|
||||||
|
*.log
|
||||||
|
*.tmp
|
||||||
|
*.bak
|
||||||
|
*.backup
|
||||||
|
*.orig
|
||||||
|
~$*
|
||||||
178
README.md
Normal file
178
README.md
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
# scientific-viz
|
||||||
|
|
||||||
|
Claude Code skill for **publication-quality scientific visualizations** — charts (matplotlib/seaborn), neural-network architecture diagrams (matplotlib patches *or* Mermaid), block-scheme flowcharts, training-pipeline diagrams, and benchmark comparison figures. Tuned for the CVGL / NADEZHDA project but reusable for any deep-learning paper.
|
||||||
|
|
||||||
|
> Behaviour spec — see [`SKILL.md`](SKILL.md). This file is the human-facing entry point.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- Need a figure for a paper / report / slide deck and want it consistent with project style.
|
||||||
|
- Want a Pareto plot (params vs R@1), grouped bar, radar, heatmap, or training curve.
|
||||||
|
- Need a Teacher–Student / multi-modal fusion architecture diagram.
|
||||||
|
- Need a Mermaid flowchart embeddable in Obsidian for an LUPI / KD pipeline.
|
||||||
|
- Want a quick comparison figure across CVGL benchmarks (University-1652, GeoText-1652, GTA-UAV, VisLoc).
|
||||||
|
|
||||||
|
## Invocation
|
||||||
|
|
||||||
|
```text
|
||||||
|
/scientific-viz <type> "<description>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Where `<type> ∈ {chart, architecture, flowchart, comparison, pipeline}`.
|
||||||
|
|
||||||
|
**Examples**
|
||||||
|
|
||||||
|
```text
|
||||||
|
/scientific-viz chart "R@1 comparison across methods on University-1652"
|
||||||
|
/scientific-viz architecture "Teacher-Student LUPI pipeline"
|
||||||
|
/scientific-viz flowchart "Training progressive staging 3 phases"
|
||||||
|
/scientific-viz comparison "Backbone candidates: params vs R@1"
|
||||||
|
/scientific-viz pipeline "LUPI distillation flow with 7 losses"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Five visualisation types
|
||||||
|
|
||||||
|
| Type | Output | Use for |
|
||||||
|
|------|--------|---------|
|
||||||
|
| `chart` | Python script (`.py`) **+** `.png` (300 DPI) **+** `.pdf` (vector) | metric comparisons, ablations, distributions, training curves |
|
||||||
|
| `architecture` | matplotlib patches Python (complex) **or** Mermaid (Obsidian-embeddable) | Teacher/Student, backbone stages, fusion modules, head designs |
|
||||||
|
| `flowchart` | Mermaid (`graph TD` / `graph LR`) + optional Python | training/inference pipeline, experimental workflow, data processing |
|
||||||
|
| `comparison` | Python `.py` + `.png/.pdf` | Pareto front, grouped bar, radar/spider, heatmap |
|
||||||
|
| `pipeline` | Mermaid + matplotlib | LUPI distillation flow, augmentation chain, edge deployment |
|
||||||
|
|
||||||
|
## Publication-quality defaults (charts)
|
||||||
|
|
||||||
|
Every generated `chart` / `comparison` / `pipeline` Python script starts with this rcParams block:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.rcParams.update({
|
||||||
|
'font.family': 'serif',
|
||||||
|
'font.size': 11,
|
||||||
|
'axes.labelsize': 12,
|
||||||
|
'axes.titlesize': 13,
|
||||||
|
'legend.fontsize': 10,
|
||||||
|
'xtick.labelsize': 10,
|
||||||
|
'ytick.labelsize': 10,
|
||||||
|
'figure.dpi': 300,
|
||||||
|
'savefig.dpi': 300,
|
||||||
|
'savefig.bbox': 'tight',
|
||||||
|
'axes.grid': True,
|
||||||
|
'grid.alpha': 0.3,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
Hard requirements baked in:
|
||||||
|
|
||||||
|
- ≥ 300 DPI, vector PDF + raster PNG saved side-by-side.
|
||||||
|
- Times New Roman / DejaVu Serif, ≥ 10 pt.
|
||||||
|
- Colorblind-safe palette (Okabe-Ito or `tab10`).
|
||||||
|
- English axis labels for publications, optional Russian for internal reports.
|
||||||
|
- Light-gray grid (`alpha=0.3`), tight layout, no clipped labels.
|
||||||
|
|
||||||
|
## NADEZHDA project palette (consistent across figures)
|
||||||
|
|
||||||
|
| Component | Colour | Hex |
|
||||||
|
|-----------|--------|-----|
|
||||||
|
| Teacher | Deep blue | `#1f77b4` |
|
||||||
|
| Student | Orange | `#ff7f0e` |
|
||||||
|
| Satellite modality | Green | `#2ca02c` |
|
||||||
|
| Drone modality | Red | `#d62728` |
|
||||||
|
| Street-view | Purple | `#9467bd` |
|
||||||
|
| Depth | Brown | `#8c564b` |
|
||||||
|
| Text | Pink | `#e377c2` |
|
||||||
|
| Loss / gradient | Gray | `#7f7f7f` |
|
||||||
|
| Edge / Jetson | Teal | `#17becf` |
|
||||||
|
|
||||||
|
## Mermaid conventions (architecture / flowchart / pipeline)
|
||||||
|
|
||||||
|
- `graph TD` — vertical flow (Teacher → Student stack).
|
||||||
|
- `graph LR` — horizontal pipeline (input → backbone → neck → heads).
|
||||||
|
- Thick arrows = main data flow · dashed arrows = gradient / loss signals.
|
||||||
|
- Loss labels on arrows: `L_task`, `L_LUPI`, `L_feat`, `L_RKD`, `L_seg`, `L_CVD_MI`, `L_CVD_Recon`.
|
||||||
|
- `style` / `classDef` to mark Teacher (blue), Student (orange), shared (green).
|
||||||
|
|
||||||
|
Ready-to-use templates: [`templates/architecture_nadezhda.md`](templates/architecture_nadezhda.md).
|
||||||
|
|
||||||
|
## Standard tensor shapes (for architecture annotations)
|
||||||
|
|
||||||
|
```
|
||||||
|
Input: [B, 3, 256, 256]
|
||||||
|
Stage 1: [B, 32, 96, 96]
|
||||||
|
Stage 2: [B, 64, 48, 48]
|
||||||
|
Stage 3: [B, 128, 24, 24]
|
||||||
|
Stage 4: [B, 256, 12, 12]
|
||||||
|
Descriptor: [B, 512] L2-normalized
|
||||||
|
```
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
Drop this repo into your vault's Claude Code skills directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://git.lissad.keenetic.name/Pikaliov/scientific-viz.git \
|
||||||
|
.claude/skills/scientific-viz
|
||||||
|
```
|
||||||
|
|
||||||
|
Or as a submodule:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git submodule add https://git.lissad.keenetic.name/Pikaliov/scientific-viz.git \
|
||||||
|
.claude/skills/scientific-viz
|
||||||
|
```
|
||||||
|
|
||||||
|
Claude Code picks up the skill on the next session. Verify with `/help` — `/scientific-viz` should appear in the user-invocable list.
|
||||||
|
|
||||||
|
**Runtime dependencies** (charts & comparison types):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install matplotlib seaborn numpy
|
||||||
|
```
|
||||||
|
|
||||||
|
Mermaid output requires no Python — the diagram block embeds directly into Obsidian, GitHub-flavoured Markdown, or Quarto.
|
||||||
|
|
||||||
|
## File layout
|
||||||
|
|
||||||
|
```
|
||||||
|
scientific-viz/
|
||||||
|
├── README.md — this file
|
||||||
|
├── SKILL.md — behaviour spec (5 types, generation pipeline)
|
||||||
|
├── reference/
|
||||||
|
│ └── chart_patterns.md — canonical Pareto/grouped-bar/radar/heatmap snippets
|
||||||
|
└── templates/
|
||||||
|
└── architecture_nadezhda.md — Mermaid templates for Teacher-Student / LUPI / fusion
|
||||||
|
```
|
||||||
|
|
||||||
|
## Hard constraints
|
||||||
|
|
||||||
|
- ❌ Никаких внешних API (OpenRouter, Gemini и т. п.) — только локальные библиотеки.
|
||||||
|
- ❌ Никакой растровой генерации архитектур через `PIL.ImageDraw` — только matplotlib `patches` или Mermaid.
|
||||||
|
- ✅ Всегда сохранять и `.py` скрипт, и результат (`.png`/`.pdf`) — рядом, рядом с заметкой / в `attachments/figures/`.
|
||||||
|
- ✅ Каждое значение в графике должно иметь источник (комментарий в коде или caption под фигурой).
|
||||||
|
- ✅ Для Obsidian — Mermaid-блок встраивается прямо в `.md`, без ссылок на внешние файлы.
|
||||||
|
- ✅ Для публикаций — английский язык подписей; для внутренних отчётов — RU допустим.
|
||||||
|
|
||||||
|
## Worked example
|
||||||
|
|
||||||
|
```text
|
||||||
|
/scientific-viz comparison "Pareto front: backbone params vs R@1 on University-1652, mark NADEZHDA target at 8.5M"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output:
|
||||||
|
|
||||||
|
- `pareto_params_r1.py` — full matplotlib script with palette, grid, edge-budget shaded region, NADEZHDA marker as star.
|
||||||
|
- `pareto_params_r1.png` (300 DPI) + `pareto_params_r1.pdf` (vector) saved next to the script.
|
||||||
|
- Inline annotation list (Sample4Geo, VimGeo, QDFL, MobileGeo, (MGS)², NADEZHDA target).
|
||||||
|
- Source comment at the top of the script linking to `1_lit_research/СИНТЕЗ_всех_статей_для_LUPI_CVGL.md` rows.
|
||||||
|
|
||||||
|
See [`reference/chart_patterns.md`](reference/chart_patterns.md) for the canonical pattern this expands from.
|
||||||
|
|
||||||
|
## Allowed tools
|
||||||
|
|
||||||
|
`Read`, `Write`, `Edit`, `Bash`, `Glob`, `Grep` — the skill writes Python scripts and Mermaid blocks directly into the vault, then optionally executes the script via `Bash` to produce the rendered `.png`/`.pdf`.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- `/analyze-paper` — surfaces the numbers this skill plots.
|
||||||
|
- `/synthesize-review` — produces the cross-paper tables that feed `comparison` figures.
|
||||||
|
- `/generate-hypothesis` — the hypotheses whose evidence is plotted.
|
||||||
181
SKILL.md
Normal file
181
SKILL.md
Normal file
@@ -0,0 +1,181 @@
|
|||||||
|
---
|
||||||
|
name: scientific-viz
|
||||||
|
description: "Generate publication-quality scientific visualizations: charts (matplotlib/seaborn), neural network architecture diagrams, block-scheme flowcharts (mermaid), training pipeline diagrams, and benchmark comparison tables. Use when the user needs any visual figure for papers, reports, or presentations."
|
||||||
|
argument-hint: "[type: chart|architecture|flowchart|comparison|pipeline] [description]"
|
||||||
|
user-invocable: true
|
||||||
|
allowed-tools: Read Write Edit Bash Glob Grep
|
||||||
|
---
|
||||||
|
|
||||||
|
# Научная визуализация для CVGL
|
||||||
|
|
||||||
|
Генерация визуализаций для научных публикаций, отчётов и презентаций проекта NADEZHDA.
|
||||||
|
|
||||||
|
## Входные данные
|
||||||
|
|
||||||
|
- `$ARGUMENTS` — тип визуализации + описание
|
||||||
|
- Примеры:
|
||||||
|
- `/scientific-viz chart "R@1 comparison across methods on University-1652"`
|
||||||
|
- `/scientific-viz architecture "Teacher-Student LUPI pipeline"`
|
||||||
|
- `/scientific-viz flowchart "Training progressive staging 3 phases"`
|
||||||
|
- `/scientific-viz comparison "Backbone candidates: params vs R@1"`
|
||||||
|
|
||||||
|
## Типы визуализаций
|
||||||
|
|
||||||
|
### 1. `chart` — Графики и диаграммы (matplotlib + seaborn)
|
||||||
|
|
||||||
|
Для: сравнение метрик, аблации, распределения, training curves.
|
||||||
|
|
||||||
|
**Выход:** Python-скрипт `.py` + сохранённый `.png`/`.pdf`
|
||||||
|
|
||||||
|
Обязательные требования:
|
||||||
|
- Разрешение ≥ 300 DPI
|
||||||
|
- Шрифт: Times New Roman или DejaVu Serif, ≥ 10pt
|
||||||
|
- Colorblind-safe палитра (Okabe-Ito или tab10)
|
||||||
|
- Подписи осей на **английском** (для публикаций)
|
||||||
|
- Легенда без перекрытия данных
|
||||||
|
- Grid: light gray, alpha=0.3
|
||||||
|
- Tight layout, без обрезки подписей
|
||||||
|
- Формат сохранения: PNG (300 DPI) + PDF (vector)
|
||||||
|
|
||||||
|
Стиль matplotlib (использовать в начале каждого скрипта):
|
||||||
|
```python
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.rcParams.update({
|
||||||
|
'font.family': 'serif',
|
||||||
|
'font.size': 11,
|
||||||
|
'axes.labelsize': 12,
|
||||||
|
'axes.titlesize': 13,
|
||||||
|
'legend.fontsize': 10,
|
||||||
|
'xtick.labelsize': 10,
|
||||||
|
'ytick.labelsize': 10,
|
||||||
|
'figure.dpi': 300,
|
||||||
|
'savefig.dpi': 300,
|
||||||
|
'savefig.bbox': 'tight',
|
||||||
|
'axes.grid': True,
|
||||||
|
'grid.alpha': 0.3,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
Шаблоны: см. [reference/chart_patterns.md](reference/chart_patterns.md)
|
||||||
|
|
||||||
|
### 2. `architecture` — Архитектуры нейросетей
|
||||||
|
|
||||||
|
Для: Teacher/Student диаграммы, backbone stages, fusion modules, head designs.
|
||||||
|
|
||||||
|
**Два варианта выхода:**
|
||||||
|
|
||||||
|
**A) Python (matplotlib + patches/arrows)** — для сложных кастомных диаграмм:
|
||||||
|
- Прямоугольники = слои/модули
|
||||||
|
- Стрелки = потоки данных
|
||||||
|
- Цвета = разные модальности/компоненты
|
||||||
|
- Аннотации: размерности тензоров, число параметров
|
||||||
|
- Используй `matplotlib.patches.FancyBboxPatch` для красивых блоков
|
||||||
|
|
||||||
|
**B) Mermaid** — для встраивания в Obsidian:
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[Satellite RGB] --> B[DINOv2-L Encoder]
|
||||||
|
C[Drone RGB] --> B
|
||||||
|
D[Depth Map] --> E[CNN Encoder]
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Шаблоны: см. [templates/architecture_nadezhda.md](templates/architecture_nadezhda.md)
|
||||||
|
|
||||||
|
### 3. `flowchart` — Блок-схемы процессов
|
||||||
|
|
||||||
|
Для: training pipeline, inference pipeline, experimental workflow, data processing.
|
||||||
|
|
||||||
|
**Выход:** Mermaid-диаграмма (встраиваемая в Obsidian) + опционально Python-версия
|
||||||
|
|
||||||
|
Mermaid-конвенции:
|
||||||
|
- `graph TD` для вертикальных flow
|
||||||
|
- `graph LR` для горизонтальных pipeline
|
||||||
|
- Цвета через `style` или `classDef`
|
||||||
|
- Подписи на английском (для публикаций) или русском (для отчётов)
|
||||||
|
|
||||||
|
### 4. `comparison` — Сравнительные визуализации
|
||||||
|
|
||||||
|
Для: scatter plot params-vs-accuracy, grouped bar chart методов, radar chart по критериям.
|
||||||
|
|
||||||
|
Паттерны:
|
||||||
|
- **Pareto front:** params (x) vs R@1 (y), маркеры = разные методы, размер = FLOPs
|
||||||
|
- **Grouped bar:** методы × датасеты, hatching для edge/cloud
|
||||||
|
- **Radar/spider:** критерии (accuracy, speed, size, generalization, robustness)
|
||||||
|
- **Heatmap:** метод × датасет → R@1 значения
|
||||||
|
|
||||||
|
### 5. `pipeline` — Диаграммы пайплайна
|
||||||
|
|
||||||
|
Для: LUPI distillation flow, data augmentation pipeline, edge deployment chain.
|
||||||
|
|
||||||
|
Специальная нотация:
|
||||||
|
- Thick arrows = main data flow
|
||||||
|
- Dashed arrows = gradient flow / loss signals
|
||||||
|
- Color: blue = Teacher, orange = Student, green = shared
|
||||||
|
- Labels on arrows: loss names (L_task, L_LUPI, L_feat, L_RKD)
|
||||||
|
|
||||||
|
## Процесс генерации
|
||||||
|
|
||||||
|
### Шаг 1: Определи тип и параметры
|
||||||
|
|
||||||
|
Из `$ARGUMENTS` извлеки:
|
||||||
|
- Тип: chart / architecture / flowchart / comparison / pipeline
|
||||||
|
- Что визуализировать
|
||||||
|
- Целевой формат: Obsidian (mermaid) / публикация (matplotlib) / оба
|
||||||
|
- Язык подписей: EN (default для публикаций) / RU (для отчётов)
|
||||||
|
|
||||||
|
### Шаг 2: Найди данные
|
||||||
|
|
||||||
|
Если визуализация требует числовых данных:
|
||||||
|
- Поищи в `1_lit_research/СИНТЕЗ_всех_статей_для_LUPI_CVGL.md`
|
||||||
|
- Или в конкретных конспектах `1_lit_research/6_cvgl/P{N}_*.md`
|
||||||
|
- Или спроси пользователя
|
||||||
|
|
||||||
|
### Шаг 3: Сгенерируй код
|
||||||
|
|
||||||
|
- Python-скрипт: сохранить в `3_work/` или `attachments/figures/`
|
||||||
|
- Mermaid: вставить inline в .md файл
|
||||||
|
- Всегда указывай: `plt.savefig("filename.png", dpi=300, bbox_inches='tight')`
|
||||||
|
|
||||||
|
### Шаг 4: Добавь аннотации
|
||||||
|
|
||||||
|
Каждая визуализация должна иметь:
|
||||||
|
- Заголовок (title)
|
||||||
|
- Подписи осей (xlabel, ylabel)
|
||||||
|
- Легенду (если >1 series)
|
||||||
|
- Источник данных (caption или комментарий в коде)
|
||||||
|
|
||||||
|
## Специфика проекта NADEZHDA
|
||||||
|
|
||||||
|
### Цветовая схема компонентов
|
||||||
|
|
||||||
|
| Компонент | Цвет | Hex |
|
||||||
|
|:----------|:-----|:----|
|
||||||
|
| Teacher | Deep blue | `#1f77b4` |
|
||||||
|
| Student | Orange | `#ff7f0e` |
|
||||||
|
| Satellite modality | Green | `#2ca02c` |
|
||||||
|
| Drone modality | Red | `#d62728` |
|
||||||
|
| Street-view | Purple | `#9467bd` |
|
||||||
|
| Depth | Brown | `#8c564b` |
|
||||||
|
| Text | Pink | `#e377c2` |
|
||||||
|
| Loss / gradient | Gray | `#7f7f7f` |
|
||||||
|
| Edge / Jetson | Teal | `#17becf` |
|
||||||
|
|
||||||
|
### Стандартные размерности тензоров
|
||||||
|
|
||||||
|
```
|
||||||
|
Input: [B, 3, 256, 256]
|
||||||
|
Stage 1: [B, 32, 96, 96]
|
||||||
|
Stage 2: [B, 64, 48, 48]
|
||||||
|
Stage 3: [B, 128, 24, 24]
|
||||||
|
Stage 4: [B, 256, 12, 12]
|
||||||
|
Descriptor: [B, 512] L2-normalized
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ограничения
|
||||||
|
|
||||||
|
- НЕ используй внешние API (OpenRouter, Gemini) — только локальные библиотеки
|
||||||
|
- НЕ генерируй растровые диаграммы архитектур через PIL/ImageDraw — используй matplotlib patches или mermaid
|
||||||
|
- Всегда сохраняй и .py скрипт, и результат (.png/.pdf)
|
||||||
|
- Для Obsidian: mermaid блоки встраиваются напрямую в .md
|
||||||
171
reference/chart_patterns.md
Normal file
171
reference/chart_patterns.md
Normal file
@@ -0,0 +1,171 @@
|
|||||||
|
# Шаблоны визуализаций для CVGL
|
||||||
|
|
||||||
|
## 1. Pareto Front: Params vs R@1
|
||||||
|
|
||||||
|
```python
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
methods = {
|
||||||
|
'Sample4Geo': (88.5, 92.65),
|
||||||
|
'VimGeo': (50.87, 96.19),
|
||||||
|
'QDFL': (5.0, 95.00),
|
||||||
|
'MobileGeo': (28.57, 97.15),
|
||||||
|
'(MGS)²': (100.0, 97.50),
|
||||||
|
'NADEZHDA': (8.5, None), # target
|
||||||
|
}
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 5))
|
||||||
|
for name, (params, r1) in methods.items():
|
||||||
|
if r1 is None:
|
||||||
|
ax.scatter(params, 90, marker='*', s=200, c='#ff7f0e', zorder=5)
|
||||||
|
ax.annotate(f'{name}\n(target)', (params, 90), fontsize=9, ha='center', va='bottom')
|
||||||
|
else:
|
||||||
|
ax.scatter(params, r1, s=100, zorder=4)
|
||||||
|
ax.annotate(name, (params, r1), fontsize=9, textcoords='offset points', xytext=(5, 5))
|
||||||
|
|
||||||
|
ax.set_xlabel('Parameters (M)')
|
||||||
|
ax.set_ylabel('R@1 (%) on University-1652')
|
||||||
|
ax.set_title('Accuracy vs Model Size: CVGL Methods')
|
||||||
|
ax.axvspan(0, 10, alpha=0.1, color='green', label='Edge budget (≤10M)')
|
||||||
|
ax.legend()
|
||||||
|
plt.savefig('pareto_params_r1.png', dpi=300, bbox_inches='tight')
|
||||||
|
plt.savefig('pareto_params_r1.pdf', bbox_inches='tight')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Grouped Bar: R@1 Across Datasets
|
||||||
|
|
||||||
|
```python
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
methods = ['Sample4Geo', 'VimGeo', 'GeoDTR+', 'CAMP', 'CGSI']
|
||||||
|
datasets = ['Univ-1652', 'CVUSA', 'CVACT-test']
|
||||||
|
data = np.array([
|
||||||
|
[92.65, 98.68, 75.0], # Sample4Geo
|
||||||
|
[0, 96.19, 81.69], # VimGeo
|
||||||
|
[94.67, 95.43, 0], # GeoDTR+
|
||||||
|
[94.46, 98.97, 0], # CAMP
|
||||||
|
[95.45, 0, 0], # CGSI
|
||||||
|
])
|
||||||
|
|
||||||
|
x = np.arange(len(methods))
|
||||||
|
width = 0.25
|
||||||
|
fig, ax = plt.subplots(figsize=(10, 5))
|
||||||
|
for i, ds in enumerate(datasets):
|
||||||
|
mask = data[:, i] > 0
|
||||||
|
bars = ax.bar(x[mask] + i * width, data[mask, i], width, label=ds)
|
||||||
|
|
||||||
|
ax.set_ylabel('R@1 (%)')
|
||||||
|
ax.set_xticks(x + width)
|
||||||
|
ax.set_xticklabels(methods, rotation=15, ha='right')
|
||||||
|
ax.legend()
|
||||||
|
ax.set_ylim(70, 102)
|
||||||
|
plt.savefig('r1_comparison.png', dpi=300, bbox_inches='tight')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3. Training Curve with Loss Components
|
||||||
|
|
||||||
|
```python
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
epochs = np.arange(60)
|
||||||
|
# Simulated progressive staging
|
||||||
|
l_task = 1.5 * np.exp(-0.03 * epochs) + 0.2
|
||||||
|
l_lupi = np.where(epochs < 10, 0, np.where(epochs < 20,
|
||||||
|
0.5 * (epochs - 10) / 10, 0.5)) * np.exp(-0.02 * (epochs - 10).clip(0))
|
||||||
|
l_feat = np.where(epochs < 10, 0, 0.3 * np.exp(-0.02 * (epochs - 10)))
|
||||||
|
l_total = l_task + l_lupi + l_feat
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4))
|
||||||
|
ax.plot(epochs, l_task, label='$L_{task}$ (InfoNCE)', linewidth=2)
|
||||||
|
ax.plot(epochs, l_lupi, label='$L_{LUPI}$ (MSE)', linewidth=2, linestyle='--')
|
||||||
|
ax.plot(epochs, l_feat, label='$L_{feat}$ (alignment)', linewidth=2, linestyle=':')
|
||||||
|
ax.plot(epochs, l_total, label='$L_{total}$', linewidth=2.5, color='black', alpha=0.7)
|
||||||
|
|
||||||
|
ax.axvspan(0, 10, alpha=0.08, color='blue', label='Warmup')
|
||||||
|
ax.axvspan(10, 20, alpha=0.08, color='orange', label='Ramp-up')
|
||||||
|
ax.axvspan(20, 60, alpha=0.05, color='green', label='Full')
|
||||||
|
|
||||||
|
ax.set_xlabel('Epoch')
|
||||||
|
ax.set_ylabel('Loss')
|
||||||
|
ax.set_title('Progressive Loss Staging (NADEZHDA)')
|
||||||
|
ax.legend(ncol=2, fontsize=9)
|
||||||
|
plt.savefig('training_curve.png', dpi=300, bbox_inches='tight')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Architecture Block Diagram (matplotlib patches)
|
||||||
|
|
||||||
|
```python
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import matplotlib.patches as mpatches
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(14, 6))
|
||||||
|
ax.set_xlim(0, 14)
|
||||||
|
ax.set_ylim(0, 6)
|
||||||
|
ax.axis('off')
|
||||||
|
|
||||||
|
def add_block(ax, xy, w, h, label, color, sublabel=''):
|
||||||
|
box = mpatches.FancyBboxPatch(xy, w, h, boxstyle='round,pad=0.1',
|
||||||
|
facecolor=color, edgecolor='black', linewidth=1.5, alpha=0.85)
|
||||||
|
ax.add_patch(box)
|
||||||
|
ax.text(xy[0] + w/2, xy[1] + h/2, label, ha='center', va='center',
|
||||||
|
fontsize=10, fontweight='bold')
|
||||||
|
if sublabel:
|
||||||
|
ax.text(xy[0] + w/2, xy[1] + 0.15, sublabel, ha='center', va='bottom',
|
||||||
|
fontsize=7, color='gray')
|
||||||
|
|
||||||
|
def add_arrow(ax, start, end, label=''):
|
||||||
|
ax.annotate('', xy=end, xytext=start,
|
||||||
|
arrowprops=dict(arrowstyle='->', lw=1.5, color='#333'))
|
||||||
|
if label:
|
||||||
|
mid = ((start[0]+end[0])/2, (start[1]+end[1])/2 + 0.15)
|
||||||
|
ax.text(*mid, label, fontsize=7, ha='center', color='#555')
|
||||||
|
|
||||||
|
# Teacher side
|
||||||
|
add_block(ax, (0.5, 4), 2, 1.2, 'DINOv2-L\n(frozen+LoRA)', '#1f77b4', '304M params')
|
||||||
|
add_block(ax, (0.5, 2), 2, 1.2, 'Multi-FiLM\nFusion', '#9467bd', 'text→visual modulation')
|
||||||
|
add_block(ax, (0.5, 0.3), 2, 1.2, 'Teacher\nDescriptor', '#1f77b4', '512-dim')
|
||||||
|
|
||||||
|
# Student side
|
||||||
|
add_block(ax, (5, 4), 2, 1.2, 'FastViT-T12\n(shared)', '#ff7f0e', '8.5M params')
|
||||||
|
add_block(ax, (5, 0.3), 2, 1.2, 'Student\nDescriptor', '#ff7f0e', '512-dim')
|
||||||
|
|
||||||
|
# Distillation arrows
|
||||||
|
add_arrow(ax, (2.5, 0.9), (5, 0.9), '$L_{LUPI}$')
|
||||||
|
add_arrow(ax, (2.5, 4.6), (5, 4.6), '$L_{feat}$')
|
||||||
|
|
||||||
|
ax.set_title('NADEZHDA: Teacher-Student Architecture', fontsize=14, fontweight='bold')
|
||||||
|
plt.savefig('nadezhda_architecture.png', dpi=300, bbox_inches='tight')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 5. Mermaid: LUPI Distillation Pipeline
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
subgraph Teacher ["Teacher (356M, cloud)"]
|
||||||
|
S1[Satellite RGB] --> TE[DINOv2-L + LoRA]
|
||||||
|
D1[Drone RGB] --> TE
|
||||||
|
SV[Street-View] --> TE
|
||||||
|
DP[Depth Map] --> TE
|
||||||
|
TX[Text Desc.] --> FM[Multi-FiLM]
|
||||||
|
FM --> TE
|
||||||
|
TE --> TD[Teacher Descriptor 512-dim]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Student ["Student (8.5M, edge)"]
|
||||||
|
S2[Satellite RGB] --> SE[FastViT-T12]
|
||||||
|
D2[Drone RGB] --> SE
|
||||||
|
SE --> SD[Student Descriptor 512-dim]
|
||||||
|
end
|
||||||
|
|
||||||
|
TD -->|L_LUPI: MSE| SD
|
||||||
|
TE -->|L_feat: Conv1x1 proj| SE
|
||||||
|
TD -->|L_RKD: relational| SD
|
||||||
|
|
||||||
|
style Teacher fill:#e3f2fd,stroke:#1565c0
|
||||||
|
style Student fill:#fff3e0,stroke:#e65100
|
||||||
|
```
|
||||||
|
````
|
||||||
115
templates/architecture_nadezhda.md
Normal file
115
templates/architecture_nadezhda.md
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
# Шаблоны архитектурных диаграмм NADEZHDA
|
||||||
|
|
||||||
|
## Mermaid: Полная система Teacher-Student
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```mermaid
|
||||||
|
graph TB
|
||||||
|
subgraph Training ["ОБУЧЕНИЕ (5 модальностей, облако)"]
|
||||||
|
direction TB
|
||||||
|
SAT_T[🛰 Satellite RGB] --> DINO[DINOv3-L<br/>~300M frozen, patch 16, RegTokens]
|
||||||
|
DRONE_T[🚁 Drone RGB] --> DINO
|
||||||
|
SV[📷 Street-View] --> DINO
|
||||||
|
DEPTH[🗺 Depth Map<br/>DA V2] --> DINO
|
||||||
|
TEXT[📝 Text<br/>MobileCLIP2] --> FILM[Multi-FiLM<br/>γ·F + β]
|
||||||
|
FILM --> DINO
|
||||||
|
|
||||||
|
DINO --> TFEAT[Teacher Features]
|
||||||
|
TFEAT --> TDESC[Teacher Descriptor<br/>512-dim, L2-norm]
|
||||||
|
TFEAT --> TSEG[OV-Seg Head]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Inference ["ИНФЕРЕНС (2 модальности, Jetson)"]
|
||||||
|
direction TB
|
||||||
|
SAT_S[🛰 Satellite RGB] --> FVIT[FastViT-T12<br/>8.5M, weight-shared]
|
||||||
|
DRONE_S[🚁 Drone RGB] --> FVIT
|
||||||
|
FVIT --> SFEAT[Student Features]
|
||||||
|
SFEAT --> CVD[CVD Split]
|
||||||
|
CVD --> SDESC[Student Descriptor<br/>512-dim, L2-norm]
|
||||||
|
end
|
||||||
|
|
||||||
|
TDESC -.->|L_LUPI: MSE| SDESC
|
||||||
|
TFEAT -.->|L_feat: Conv1×1| SFEAT
|
||||||
|
TDESC -.->|L_RKD: relational| SDESC
|
||||||
|
TSEG -.->|L_seg: CE+KL| SFEAT
|
||||||
|
|
||||||
|
style Training fill:#e8eaf6,stroke:#283593
|
||||||
|
style Inference fill:#fff8e1,stroke:#f57f17
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
## Mermaid: Progressive Loss Staging
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```mermaid
|
||||||
|
gantt
|
||||||
|
title Progressive Loss Staging (60 epochs)
|
||||||
|
dateFormat X
|
||||||
|
axisFormat %s
|
||||||
|
|
||||||
|
section Losses
|
||||||
|
L_task (InfoNCE) :active, 0, 60
|
||||||
|
L_feat (alignment) :crit, 10, 60
|
||||||
|
L_CRD (contrastive repr.) :crit, 10, 60
|
||||||
|
L_LUPI (privileged MSE) :done, 20, 60
|
||||||
|
L_RKD (relational) :done, 20, 60
|
||||||
|
L_seg (segmentation) :done, 20, 60
|
||||||
|
L_CVD (disentanglement) :done, 20, 60
|
||||||
|
|
||||||
|
section Phases
|
||||||
|
Warmup (L_task only) :milestone, 0, 10
|
||||||
|
Ramp-up (+L_feat, L_CRD) :milestone, 10, 20
|
||||||
|
Full (all 7 losses) :milestone, 20, 60
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
## Mermaid: Student Backbone Stages
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
IN["Input<br/>[B,3,256,256]"] --> S1["Stage 1<br/>[B,32,96,96]"]
|
||||||
|
S1 --> S2["Stage 2<br/>[B,64,48,48]"]
|
||||||
|
S2 --> BRIDGE["Conv1×1<br/>Bridge"]
|
||||||
|
BRIDGE --> S3["Stage 3<br/>[B,128,24,24]"]
|
||||||
|
S3 --> S4["Stage 4<br/>[B,256,12,12]"]
|
||||||
|
S4 --> POOL["GGeM Pool"]
|
||||||
|
POOL --> DESC["Descriptor<br/>[B,512]"]
|
||||||
|
|
||||||
|
S1 ~~~ NOTE1["SOFIA<br/>(DCN blocks)"]
|
||||||
|
S3 ~~~ NOTE2["MambaVision<br/>(SSM+ViT)"]
|
||||||
|
|
||||||
|
style S1 fill:#fff3e0,stroke:#e65100
|
||||||
|
style S2 fill:#fff3e0,stroke:#e65100
|
||||||
|
style S3 fill:#e3f2fd,stroke:#1565c0
|
||||||
|
style S4 fill:#e3f2fd,stroke:#1565c0
|
||||||
|
style NOTE1 fill:#fff3e0,stroke:none,color:#e65100
|
||||||
|
style NOTE2 fill:#e3f2fd,stroke:none,color:#1565c0
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
## Mermaid: Data Augmentation Pipeline
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
RAW[Raw RGB<br/>512×512] --> RESIZE[Resize<br/>256×256]
|
||||||
|
RESIZE --> DA["Depth Anything V2<br/>Large (335M)"]
|
||||||
|
RESIZE --> DSINE["DSINE<br/>(30M)"]
|
||||||
|
RESIZE --> SEG["SegFormer-B5<br/>(85M)"]
|
||||||
|
|
||||||
|
DA --> D_OUT["depth.npy<br/>[1,256,256]"]
|
||||||
|
DSINE --> N_OUT["normal.npy<br/>[3,256,256]"]
|
||||||
|
SEG --> S_OUT["seg.npy<br/>[1,256,256]"]
|
||||||
|
|
||||||
|
RESIZE --> RGB["RGB<br/>[3,256,256]"]
|
||||||
|
RGB --> CONCAT["concat_8ch.npy<br/>[8,256,256]"]
|
||||||
|
D_OUT --> CONCAT
|
||||||
|
N_OUT --> CONCAT
|
||||||
|
S_OUT --> CONCAT
|
||||||
|
|
||||||
|
style DA fill:#e8f5e9,stroke:#2e7d32
|
||||||
|
style DSINE fill:#fce4ec,stroke:#c62828
|
||||||
|
style SEG fill:#e3f2fd,stroke:#1565c0
|
||||||
|
```
|
||||||
|
````
|
||||||
Reference in New Issue
Block a user