Initial commit — scientific-viz skill
Claude Code skill for publication-quality scientific visualisations: chart, architecture, flowchart, comparison, pipeline. Bundles matplotlib publication defaults (300 DPI, serif font, colorblind palette), NADEZHDA project palette, Mermaid conventions, and ready-to-use templates. Contents: - SKILL.md — behaviour spec (5 types, generation pipeline) - README.md — human-facing entry: when, install, examples - reference/chart_patterns.md — Pareto/bar/radar/heatmap snippets - templates/architecture_nadezhda.md — Mermaid templates for Teacher-Student / LUPI / fusion Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
58
.gitignore
vendored
Normal file
58
.gitignore
vendored
Normal file
@@ -0,0 +1,58 @@
|
||||
# Python build artifacts
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.egg-info/
|
||||
*.egg
|
||||
.eggs/
|
||||
build/
|
||||
dist/
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
ENV/
|
||||
.python-version
|
||||
|
||||
# Coverage / testing
|
||||
.coverage
|
||||
.coverage.*
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.dmypy.json
|
||||
|
||||
# IDE / OS
|
||||
.idea/
|
||||
.vscode/
|
||||
!.vscode/settings.json
|
||||
!.vscode/launch.json
|
||||
!.vscode/extensions.json
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
desktop.ini
|
||||
|
||||
# Jupyter
|
||||
.ipynb_checkpoints/
|
||||
|
||||
# Secrets — never commit
|
||||
.env
|
||||
.env.local
|
||||
*.key
|
||||
*.pem
|
||||
credentials.json
|
||||
|
||||
# Temp
|
||||
*.log
|
||||
*.tmp
|
||||
*.bak
|
||||
*.backup
|
||||
*.orig
|
||||
~$*
|
||||
178
README.md
Normal file
178
README.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# scientific-viz
|
||||
|
||||
Claude Code skill for **publication-quality scientific visualizations** — charts (matplotlib/seaborn), neural-network architecture diagrams (matplotlib patches *or* Mermaid), block-scheme flowcharts, training-pipeline diagrams, and benchmark comparison figures. Tuned for the CVGL / NADEZHDA project but reusable for any deep-learning paper.
|
||||
|
||||
> Behaviour spec — see [`SKILL.md`](SKILL.md). This file is the human-facing entry point.
|
||||
|
||||
## When to use
|
||||
|
||||
- Need a figure for a paper / report / slide deck and want it consistent with project style.
|
||||
- Want a Pareto plot (params vs R@1), grouped bar, radar, heatmap, or training curve.
|
||||
- Need a Teacher–Student / multi-modal fusion architecture diagram.
|
||||
- Need a Mermaid flowchart embeddable in Obsidian for an LUPI / KD pipeline.
|
||||
- Want a quick comparison figure across CVGL benchmarks (University-1652, GeoText-1652, GTA-UAV, VisLoc).
|
||||
|
||||
## Invocation
|
||||
|
||||
```text
|
||||
/scientific-viz <type> "<description>"
|
||||
```
|
||||
|
||||
Where `<type> ∈ {chart, architecture, flowchart, comparison, pipeline}`.
|
||||
|
||||
**Examples**
|
||||
|
||||
```text
|
||||
/scientific-viz chart "R@1 comparison across methods on University-1652"
|
||||
/scientific-viz architecture "Teacher-Student LUPI pipeline"
|
||||
/scientific-viz flowchart "Training progressive staging 3 phases"
|
||||
/scientific-viz comparison "Backbone candidates: params vs R@1"
|
||||
/scientific-viz pipeline "LUPI distillation flow with 7 losses"
|
||||
```
|
||||
|
||||
## Five visualisation types
|
||||
|
||||
| Type | Output | Use for |
|
||||
|------|--------|---------|
|
||||
| `chart` | Python script (`.py`) **+** `.png` (300 DPI) **+** `.pdf` (vector) | metric comparisons, ablations, distributions, training curves |
|
||||
| `architecture` | matplotlib patches Python (complex) **or** Mermaid (Obsidian-embeddable) | Teacher/Student, backbone stages, fusion modules, head designs |
|
||||
| `flowchart` | Mermaid (`graph TD` / `graph LR`) + optional Python | training/inference pipeline, experimental workflow, data processing |
|
||||
| `comparison` | Python `.py` + `.png/.pdf` | Pareto front, grouped bar, radar/spider, heatmap |
|
||||
| `pipeline` | Mermaid + matplotlib | LUPI distillation flow, augmentation chain, edge deployment |
|
||||
|
||||
## Publication-quality defaults (charts)
|
||||
|
||||
Every generated `chart` / `comparison` / `pipeline` Python script starts with this rcParams block:
|
||||
|
||||
```python
|
||||
import matplotlib
|
||||
matplotlib.rcParams.update({
|
||||
'font.family': 'serif',
|
||||
'font.size': 11,
|
||||
'axes.labelsize': 12,
|
||||
'axes.titlesize': 13,
|
||||
'legend.fontsize': 10,
|
||||
'xtick.labelsize': 10,
|
||||
'ytick.labelsize': 10,
|
||||
'figure.dpi': 300,
|
||||
'savefig.dpi': 300,
|
||||
'savefig.bbox': 'tight',
|
||||
'axes.grid': True,
|
||||
'grid.alpha': 0.3,
|
||||
})
|
||||
```
|
||||
|
||||
Hard requirements baked in:
|
||||
|
||||
- ≥ 300 DPI, vector PDF + raster PNG saved side-by-side.
|
||||
- Times New Roman / DejaVu Serif, ≥ 10 pt.
|
||||
- Colorblind-safe palette (Okabe-Ito or `tab10`).
|
||||
- English axis labels for publications, optional Russian for internal reports.
|
||||
- Light-gray grid (`alpha=0.3`), tight layout, no clipped labels.
|
||||
|
||||
## NADEZHDA project palette (consistent across figures)
|
||||
|
||||
| Component | Colour | Hex |
|
||||
|-----------|--------|-----|
|
||||
| Teacher | Deep blue | `#1f77b4` |
|
||||
| Student | Orange | `#ff7f0e` |
|
||||
| Satellite modality | Green | `#2ca02c` |
|
||||
| Drone modality | Red | `#d62728` |
|
||||
| Street-view | Purple | `#9467bd` |
|
||||
| Depth | Brown | `#8c564b` |
|
||||
| Text | Pink | `#e377c2` |
|
||||
| Loss / gradient | Gray | `#7f7f7f` |
|
||||
| Edge / Jetson | Teal | `#17becf` |
|
||||
|
||||
## Mermaid conventions (architecture / flowchart / pipeline)
|
||||
|
||||
- `graph TD` — vertical flow (Teacher → Student stack).
|
||||
- `graph LR` — horizontal pipeline (input → backbone → neck → heads).
|
||||
- Thick arrows = main data flow · dashed arrows = gradient / loss signals.
|
||||
- Loss labels on arrows: `L_task`, `L_LUPI`, `L_feat`, `L_RKD`, `L_seg`, `L_CVD_MI`, `L_CVD_Recon`.
|
||||
- `style` / `classDef` to mark Teacher (blue), Student (orange), shared (green).
|
||||
|
||||
Ready-to-use templates: [`templates/architecture_nadezhda.md`](templates/architecture_nadezhda.md).
|
||||
|
||||
## Standard tensor shapes (for architecture annotations)
|
||||
|
||||
```
|
||||
Input: [B, 3, 256, 256]
|
||||
Stage 1: [B, 32, 96, 96]
|
||||
Stage 2: [B, 64, 48, 48]
|
||||
Stage 3: [B, 128, 24, 24]
|
||||
Stage 4: [B, 256, 12, 12]
|
||||
Descriptor: [B, 512] L2-normalized
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
Drop this repo into your vault's Claude Code skills directory:
|
||||
|
||||
```bash
|
||||
git clone https://git.lissad.keenetic.name/Pikaliov/scientific-viz.git \
|
||||
.claude/skills/scientific-viz
|
||||
```
|
||||
|
||||
Or as a submodule:
|
||||
|
||||
```bash
|
||||
git submodule add https://git.lissad.keenetic.name/Pikaliov/scientific-viz.git \
|
||||
.claude/skills/scientific-viz
|
||||
```
|
||||
|
||||
Claude Code picks up the skill on the next session. Verify with `/help` — `/scientific-viz` should appear in the user-invocable list.
|
||||
|
||||
**Runtime dependencies** (charts & comparison types):
|
||||
|
||||
```bash
|
||||
pip install matplotlib seaborn numpy
|
||||
```
|
||||
|
||||
Mermaid output requires no Python — the diagram block embeds directly into Obsidian, GitHub-flavoured Markdown, or Quarto.
|
||||
|
||||
## File layout
|
||||
|
||||
```
|
||||
scientific-viz/
|
||||
├── README.md — this file
|
||||
├── SKILL.md — behaviour spec (5 types, generation pipeline)
|
||||
├── reference/
|
||||
│ └── chart_patterns.md — canonical Pareto/grouped-bar/radar/heatmap snippets
|
||||
└── templates/
|
||||
└── architecture_nadezhda.md — Mermaid templates for Teacher-Student / LUPI / fusion
|
||||
```
|
||||
|
||||
## Hard constraints
|
||||
|
||||
- ❌ Никаких внешних API (OpenRouter, Gemini и т. п.) — только локальные библиотеки.
|
||||
- ❌ Никакой растровой генерации архитектур через `PIL.ImageDraw` — только matplotlib `patches` или Mermaid.
|
||||
- ✅ Всегда сохранять и `.py` скрипт, и результат (`.png`/`.pdf`) — рядом, рядом с заметкой / в `attachments/figures/`.
|
||||
- ✅ Каждое значение в графике должно иметь источник (комментарий в коде или caption под фигурой).
|
||||
- ✅ Для Obsidian — Mermaid-блок встраивается прямо в `.md`, без ссылок на внешние файлы.
|
||||
- ✅ Для публикаций — английский язык подписей; для внутренних отчётов — RU допустим.
|
||||
|
||||
## Worked example
|
||||
|
||||
```text
|
||||
/scientific-viz comparison "Pareto front: backbone params vs R@1 on University-1652, mark NADEZHDA target at 8.5M"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
- `pareto_params_r1.py` — full matplotlib script with palette, grid, edge-budget shaded region, NADEZHDA marker as star.
|
||||
- `pareto_params_r1.png` (300 DPI) + `pareto_params_r1.pdf` (vector) saved next to the script.
|
||||
- Inline annotation list (Sample4Geo, VimGeo, QDFL, MobileGeo, (MGS)², NADEZHDA target).
|
||||
- Source comment at the top of the script linking to `1_lit_research/СИНТЕЗ_всех_статей_для_LUPI_CVGL.md` rows.
|
||||
|
||||
See [`reference/chart_patterns.md`](reference/chart_patterns.md) for the canonical pattern this expands from.
|
||||
|
||||
## Allowed tools
|
||||
|
||||
`Read`, `Write`, `Edit`, `Bash`, `Glob`, `Grep` — the skill writes Python scripts and Mermaid blocks directly into the vault, then optionally executes the script via `Bash` to produce the rendered `.png`/`.pdf`.
|
||||
|
||||
## Related skills
|
||||
|
||||
- `/analyze-paper` — surfaces the numbers this skill plots.
|
||||
- `/synthesize-review` — produces the cross-paper tables that feed `comparison` figures.
|
||||
- `/generate-hypothesis` — the hypotheses whose evidence is plotted.
|
||||
181
SKILL.md
Normal file
181
SKILL.md
Normal file
@@ -0,0 +1,181 @@
|
||||
---
|
||||
name: scientific-viz
|
||||
description: "Generate publication-quality scientific visualizations: charts (matplotlib/seaborn), neural network architecture diagrams, block-scheme flowcharts (mermaid), training pipeline diagrams, and benchmark comparison tables. Use when the user needs any visual figure for papers, reports, or presentations."
|
||||
argument-hint: "[type: chart|architecture|flowchart|comparison|pipeline] [description]"
|
||||
user-invocable: true
|
||||
allowed-tools: Read Write Edit Bash Glob Grep
|
||||
---
|
||||
|
||||
# Научная визуализация для CVGL
|
||||
|
||||
Генерация визуализаций для научных публикаций, отчётов и презентаций проекта NADEZHDA.
|
||||
|
||||
## Входные данные
|
||||
|
||||
- `$ARGUMENTS` — тип визуализации + описание
|
||||
- Примеры:
|
||||
- `/scientific-viz chart "R@1 comparison across methods on University-1652"`
|
||||
- `/scientific-viz architecture "Teacher-Student LUPI pipeline"`
|
||||
- `/scientific-viz flowchart "Training progressive staging 3 phases"`
|
||||
- `/scientific-viz comparison "Backbone candidates: params vs R@1"`
|
||||
|
||||
## Типы визуализаций
|
||||
|
||||
### 1. `chart` — Графики и диаграммы (matplotlib + seaborn)
|
||||
|
||||
Для: сравнение метрик, аблации, распределения, training curves.
|
||||
|
||||
**Выход:** Python-скрипт `.py` + сохранённый `.png`/`.pdf`
|
||||
|
||||
Обязательные требования:
|
||||
- Разрешение ≥ 300 DPI
|
||||
- Шрифт: Times New Roman или DejaVu Serif, ≥ 10pt
|
||||
- Colorblind-safe палитра (Okabe-Ito или tab10)
|
||||
- Подписи осей на **английском** (для публикаций)
|
||||
- Легенда без перекрытия данных
|
||||
- Grid: light gray, alpha=0.3
|
||||
- Tight layout, без обрезки подписей
|
||||
- Формат сохранения: PNG (300 DPI) + PDF (vector)
|
||||
|
||||
Стиль matplotlib (использовать в начале каждого скрипта):
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib
|
||||
matplotlib.rcParams.update({
|
||||
'font.family': 'serif',
|
||||
'font.size': 11,
|
||||
'axes.labelsize': 12,
|
||||
'axes.titlesize': 13,
|
||||
'legend.fontsize': 10,
|
||||
'xtick.labelsize': 10,
|
||||
'ytick.labelsize': 10,
|
||||
'figure.dpi': 300,
|
||||
'savefig.dpi': 300,
|
||||
'savefig.bbox': 'tight',
|
||||
'axes.grid': True,
|
||||
'grid.alpha': 0.3,
|
||||
})
|
||||
```
|
||||
|
||||
Шаблоны: см. [reference/chart_patterns.md](reference/chart_patterns.md)
|
||||
|
||||
### 2. `architecture` — Архитектуры нейросетей
|
||||
|
||||
Для: Teacher/Student диаграммы, backbone stages, fusion modules, head designs.
|
||||
|
||||
**Два варианта выхода:**
|
||||
|
||||
**A) Python (matplotlib + patches/arrows)** — для сложных кастомных диаграмм:
|
||||
- Прямоугольники = слои/модули
|
||||
- Стрелки = потоки данных
|
||||
- Цвета = разные модальности/компоненты
|
||||
- Аннотации: размерности тензоров, число параметров
|
||||
- Используй `matplotlib.patches.FancyBboxPatch` для красивых блоков
|
||||
|
||||
**B) Mermaid** — для встраивания в Obsidian:
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Satellite RGB] --> B[DINOv2-L Encoder]
|
||||
C[Drone RGB] --> B
|
||||
D[Depth Map] --> E[CNN Encoder]
|
||||
...
|
||||
```
|
||||
|
||||
Шаблоны: см. [templates/architecture_nadezhda.md](templates/architecture_nadezhda.md)
|
||||
|
||||
### 3. `flowchart` — Блок-схемы процессов
|
||||
|
||||
Для: training pipeline, inference pipeline, experimental workflow, data processing.
|
||||
|
||||
**Выход:** Mermaid-диаграмма (встраиваемая в Obsidian) + опционально Python-версия
|
||||
|
||||
Mermaid-конвенции:
|
||||
- `graph TD` для вертикальных flow
|
||||
- `graph LR` для горизонтальных pipeline
|
||||
- Цвета через `style` или `classDef`
|
||||
- Подписи на английском (для публикаций) или русском (для отчётов)
|
||||
|
||||
### 4. `comparison` — Сравнительные визуализации
|
||||
|
||||
Для: scatter plot params-vs-accuracy, grouped bar chart методов, radar chart по критериям.
|
||||
|
||||
Паттерны:
|
||||
- **Pareto front:** params (x) vs R@1 (y), маркеры = разные методы, размер = FLOPs
|
||||
- **Grouped bar:** методы × датасеты, hatching для edge/cloud
|
||||
- **Radar/spider:** критерии (accuracy, speed, size, generalization, robustness)
|
||||
- **Heatmap:** метод × датасет → R@1 значения
|
||||
|
||||
### 5. `pipeline` — Диаграммы пайплайна
|
||||
|
||||
Для: LUPI distillation flow, data augmentation pipeline, edge deployment chain.
|
||||
|
||||
Специальная нотация:
|
||||
- Thick arrows = main data flow
|
||||
- Dashed arrows = gradient flow / loss signals
|
||||
- Color: blue = Teacher, orange = Student, green = shared
|
||||
- Labels on arrows: loss names (L_task, L_LUPI, L_feat, L_RKD)
|
||||
|
||||
## Процесс генерации
|
||||
|
||||
### Шаг 1: Определи тип и параметры
|
||||
|
||||
Из `$ARGUMENTS` извлеки:
|
||||
- Тип: chart / architecture / flowchart / comparison / pipeline
|
||||
- Что визуализировать
|
||||
- Целевой формат: Obsidian (mermaid) / публикация (matplotlib) / оба
|
||||
- Язык подписей: EN (default для публикаций) / RU (для отчётов)
|
||||
|
||||
### Шаг 2: Найди данные
|
||||
|
||||
Если визуализация требует числовых данных:
|
||||
- Поищи в `1_lit_research/СИНТЕЗ_всех_статей_для_LUPI_CVGL.md`
|
||||
- Или в конкретных конспектах `1_lit_research/6_cvgl/P{N}_*.md`
|
||||
- Или спроси пользователя
|
||||
|
||||
### Шаг 3: Сгенерируй код
|
||||
|
||||
- Python-скрипт: сохранить в `3_work/` или `attachments/figures/`
|
||||
- Mermaid: вставить inline в .md файл
|
||||
- Всегда указывай: `plt.savefig("filename.png", dpi=300, bbox_inches='tight')`
|
||||
|
||||
### Шаг 4: Добавь аннотации
|
||||
|
||||
Каждая визуализация должна иметь:
|
||||
- Заголовок (title)
|
||||
- Подписи осей (xlabel, ylabel)
|
||||
- Легенду (если >1 series)
|
||||
- Источник данных (caption или комментарий в коде)
|
||||
|
||||
## Специфика проекта NADEZHDA
|
||||
|
||||
### Цветовая схема компонентов
|
||||
|
||||
| Компонент | Цвет | Hex |
|
||||
|:----------|:-----|:----|
|
||||
| Teacher | Deep blue | `#1f77b4` |
|
||||
| Student | Orange | `#ff7f0e` |
|
||||
| Satellite modality | Green | `#2ca02c` |
|
||||
| Drone modality | Red | `#d62728` |
|
||||
| Street-view | Purple | `#9467bd` |
|
||||
| Depth | Brown | `#8c564b` |
|
||||
| Text | Pink | `#e377c2` |
|
||||
| Loss / gradient | Gray | `#7f7f7f` |
|
||||
| Edge / Jetson | Teal | `#17becf` |
|
||||
|
||||
### Стандартные размерности тензоров
|
||||
|
||||
```
|
||||
Input: [B, 3, 256, 256]
|
||||
Stage 1: [B, 32, 96, 96]
|
||||
Stage 2: [B, 64, 48, 48]
|
||||
Stage 3: [B, 128, 24, 24]
|
||||
Stage 4: [B, 256, 12, 12]
|
||||
Descriptor: [B, 512] L2-normalized
|
||||
```
|
||||
|
||||
## Ограничения
|
||||
|
||||
- НЕ используй внешние API (OpenRouter, Gemini) — только локальные библиотеки
|
||||
- НЕ генерируй растровые диаграммы архитектур через PIL/ImageDraw — используй matplotlib patches или mermaid
|
||||
- Всегда сохраняй и .py скрипт, и результат (.png/.pdf)
|
||||
- Для Obsidian: mermaid блоки встраиваются напрямую в .md
|
||||
171
reference/chart_patterns.md
Normal file
171
reference/chart_patterns.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Шаблоны визуализаций для CVGL
|
||||
|
||||
## 1. Pareto Front: Params vs R@1
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
methods = {
|
||||
'Sample4Geo': (88.5, 92.65),
|
||||
'VimGeo': (50.87, 96.19),
|
||||
'QDFL': (5.0, 95.00),
|
||||
'MobileGeo': (28.57, 97.15),
|
||||
'(MGS)²': (100.0, 97.50),
|
||||
'NADEZHDA': (8.5, None), # target
|
||||
}
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 5))
|
||||
for name, (params, r1) in methods.items():
|
||||
if r1 is None:
|
||||
ax.scatter(params, 90, marker='*', s=200, c='#ff7f0e', zorder=5)
|
||||
ax.annotate(f'{name}\n(target)', (params, 90), fontsize=9, ha='center', va='bottom')
|
||||
else:
|
||||
ax.scatter(params, r1, s=100, zorder=4)
|
||||
ax.annotate(name, (params, r1), fontsize=9, textcoords='offset points', xytext=(5, 5))
|
||||
|
||||
ax.set_xlabel('Parameters (M)')
|
||||
ax.set_ylabel('R@1 (%) on University-1652')
|
||||
ax.set_title('Accuracy vs Model Size: CVGL Methods')
|
||||
ax.axvspan(0, 10, alpha=0.1, color='green', label='Edge budget (≤10M)')
|
||||
ax.legend()
|
||||
plt.savefig('pareto_params_r1.png', dpi=300, bbox_inches='tight')
|
||||
plt.savefig('pareto_params_r1.pdf', bbox_inches='tight')
|
||||
```
|
||||
|
||||
## 2. Grouped Bar: R@1 Across Datasets
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
methods = ['Sample4Geo', 'VimGeo', 'GeoDTR+', 'CAMP', 'CGSI']
|
||||
datasets = ['Univ-1652', 'CVUSA', 'CVACT-test']
|
||||
data = np.array([
|
||||
[92.65, 98.68, 75.0], # Sample4Geo
|
||||
[0, 96.19, 81.69], # VimGeo
|
||||
[94.67, 95.43, 0], # GeoDTR+
|
||||
[94.46, 98.97, 0], # CAMP
|
||||
[95.45, 0, 0], # CGSI
|
||||
])
|
||||
|
||||
x = np.arange(len(methods))
|
||||
width = 0.25
|
||||
fig, ax = plt.subplots(figsize=(10, 5))
|
||||
for i, ds in enumerate(datasets):
|
||||
mask = data[:, i] > 0
|
||||
bars = ax.bar(x[mask] + i * width, data[mask, i], width, label=ds)
|
||||
|
||||
ax.set_ylabel('R@1 (%)')
|
||||
ax.set_xticks(x + width)
|
||||
ax.set_xticklabels(methods, rotation=15, ha='right')
|
||||
ax.legend()
|
||||
ax.set_ylim(70, 102)
|
||||
plt.savefig('r1_comparison.png', dpi=300, bbox_inches='tight')
|
||||
```
|
||||
|
||||
## 3. Training Curve with Loss Components
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
epochs = np.arange(60)
|
||||
# Simulated progressive staging
|
||||
l_task = 1.5 * np.exp(-0.03 * epochs) + 0.2
|
||||
l_lupi = np.where(epochs < 10, 0, np.where(epochs < 20,
|
||||
0.5 * (epochs - 10) / 10, 0.5)) * np.exp(-0.02 * (epochs - 10).clip(0))
|
||||
l_feat = np.where(epochs < 10, 0, 0.3 * np.exp(-0.02 * (epochs - 10)))
|
||||
l_total = l_task + l_lupi + l_feat
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8, 4))
|
||||
ax.plot(epochs, l_task, label='$L_{task}$ (InfoNCE)', linewidth=2)
|
||||
ax.plot(epochs, l_lupi, label='$L_{LUPI}$ (MSE)', linewidth=2, linestyle='--')
|
||||
ax.plot(epochs, l_feat, label='$L_{feat}$ (alignment)', linewidth=2, linestyle=':')
|
||||
ax.plot(epochs, l_total, label='$L_{total}$', linewidth=2.5, color='black', alpha=0.7)
|
||||
|
||||
ax.axvspan(0, 10, alpha=0.08, color='blue', label='Warmup')
|
||||
ax.axvspan(10, 20, alpha=0.08, color='orange', label='Ramp-up')
|
||||
ax.axvspan(20, 60, alpha=0.05, color='green', label='Full')
|
||||
|
||||
ax.set_xlabel('Epoch')
|
||||
ax.set_ylabel('Loss')
|
||||
ax.set_title('Progressive Loss Staging (NADEZHDA)')
|
||||
ax.legend(ncol=2, fontsize=9)
|
||||
plt.savefig('training_curve.png', dpi=300, bbox_inches='tight')
|
||||
```
|
||||
|
||||
## 4. Architecture Block Diagram (matplotlib patches)
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib.patches as mpatches
|
||||
|
||||
fig, ax = plt.subplots(figsize=(14, 6))
|
||||
ax.set_xlim(0, 14)
|
||||
ax.set_ylim(0, 6)
|
||||
ax.axis('off')
|
||||
|
||||
def add_block(ax, xy, w, h, label, color, sublabel=''):
|
||||
box = mpatches.FancyBboxPatch(xy, w, h, boxstyle='round,pad=0.1',
|
||||
facecolor=color, edgecolor='black', linewidth=1.5, alpha=0.85)
|
||||
ax.add_patch(box)
|
||||
ax.text(xy[0] + w/2, xy[1] + h/2, label, ha='center', va='center',
|
||||
fontsize=10, fontweight='bold')
|
||||
if sublabel:
|
||||
ax.text(xy[0] + w/2, xy[1] + 0.15, sublabel, ha='center', va='bottom',
|
||||
fontsize=7, color='gray')
|
||||
|
||||
def add_arrow(ax, start, end, label=''):
|
||||
ax.annotate('', xy=end, xytext=start,
|
||||
arrowprops=dict(arrowstyle='->', lw=1.5, color='#333'))
|
||||
if label:
|
||||
mid = ((start[0]+end[0])/2, (start[1]+end[1])/2 + 0.15)
|
||||
ax.text(*mid, label, fontsize=7, ha='center', color='#555')
|
||||
|
||||
# Teacher side
|
||||
add_block(ax, (0.5, 4), 2, 1.2, 'DINOv2-L\n(frozen+LoRA)', '#1f77b4', '304M params')
|
||||
add_block(ax, (0.5, 2), 2, 1.2, 'Multi-FiLM\nFusion', '#9467bd', 'text→visual modulation')
|
||||
add_block(ax, (0.5, 0.3), 2, 1.2, 'Teacher\nDescriptor', '#1f77b4', '512-dim')
|
||||
|
||||
# Student side
|
||||
add_block(ax, (5, 4), 2, 1.2, 'FastViT-T12\n(shared)', '#ff7f0e', '8.5M params')
|
||||
add_block(ax, (5, 0.3), 2, 1.2, 'Student\nDescriptor', '#ff7f0e', '512-dim')
|
||||
|
||||
# Distillation arrows
|
||||
add_arrow(ax, (2.5, 0.9), (5, 0.9), '$L_{LUPI}$')
|
||||
add_arrow(ax, (2.5, 4.6), (5, 4.6), '$L_{feat}$')
|
||||
|
||||
ax.set_title('NADEZHDA: Teacher-Student Architecture', fontsize=14, fontweight='bold')
|
||||
plt.savefig('nadezhda_architecture.png', dpi=300, bbox_inches='tight')
|
||||
```
|
||||
|
||||
## 5. Mermaid: LUPI Distillation Pipeline
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Teacher ["Teacher (356M, cloud)"]
|
||||
S1[Satellite RGB] --> TE[DINOv2-L + LoRA]
|
||||
D1[Drone RGB] --> TE
|
||||
SV[Street-View] --> TE
|
||||
DP[Depth Map] --> TE
|
||||
TX[Text Desc.] --> FM[Multi-FiLM]
|
||||
FM --> TE
|
||||
TE --> TD[Teacher Descriptor 512-dim]
|
||||
end
|
||||
|
||||
subgraph Student ["Student (8.5M, edge)"]
|
||||
S2[Satellite RGB] --> SE[FastViT-T12]
|
||||
D2[Drone RGB] --> SE
|
||||
SE --> SD[Student Descriptor 512-dim]
|
||||
end
|
||||
|
||||
TD -->|L_LUPI: MSE| SD
|
||||
TE -->|L_feat: Conv1x1 proj| SE
|
||||
TD -->|L_RKD: relational| SD
|
||||
|
||||
style Teacher fill:#e3f2fd,stroke:#1565c0
|
||||
style Student fill:#fff3e0,stroke:#e65100
|
||||
```
|
||||
````
|
||||
115
templates/architecture_nadezhda.md
Normal file
115
templates/architecture_nadezhda.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Шаблоны архитектурных диаграмм NADEZHDA
|
||||
|
||||
## Mermaid: Полная система Teacher-Student
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Training ["ОБУЧЕНИЕ (5 модальностей, облако)"]
|
||||
direction TB
|
||||
SAT_T[🛰 Satellite RGB] --> DINO[DINOv3-L<br/>~300M frozen, patch 16, RegTokens]
|
||||
DRONE_T[🚁 Drone RGB] --> DINO
|
||||
SV[📷 Street-View] --> DINO
|
||||
DEPTH[🗺 Depth Map<br/>DA V2] --> DINO
|
||||
TEXT[📝 Text<br/>MobileCLIP2] --> FILM[Multi-FiLM<br/>γ·F + β]
|
||||
FILM --> DINO
|
||||
|
||||
DINO --> TFEAT[Teacher Features]
|
||||
TFEAT --> TDESC[Teacher Descriptor<br/>512-dim, L2-norm]
|
||||
TFEAT --> TSEG[OV-Seg Head]
|
||||
end
|
||||
|
||||
subgraph Inference ["ИНФЕРЕНС (2 модальности, Jetson)"]
|
||||
direction TB
|
||||
SAT_S[🛰 Satellite RGB] --> FVIT[FastViT-T12<br/>8.5M, weight-shared]
|
||||
DRONE_S[🚁 Drone RGB] --> FVIT
|
||||
FVIT --> SFEAT[Student Features]
|
||||
SFEAT --> CVD[CVD Split]
|
||||
CVD --> SDESC[Student Descriptor<br/>512-dim, L2-norm]
|
||||
end
|
||||
|
||||
TDESC -.->|L_LUPI: MSE| SDESC
|
||||
TFEAT -.->|L_feat: Conv1×1| SFEAT
|
||||
TDESC -.->|L_RKD: relational| SDESC
|
||||
TSEG -.->|L_seg: CE+KL| SFEAT
|
||||
|
||||
style Training fill:#e8eaf6,stroke:#283593
|
||||
style Inference fill:#fff8e1,stroke:#f57f17
|
||||
```
|
||||
````
|
||||
|
||||
## Mermaid: Progressive Loss Staging
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
gantt
|
||||
title Progressive Loss Staging (60 epochs)
|
||||
dateFormat X
|
||||
axisFormat %s
|
||||
|
||||
section Losses
|
||||
L_task (InfoNCE) :active, 0, 60
|
||||
L_feat (alignment) :crit, 10, 60
|
||||
L_CRD (contrastive repr.) :crit, 10, 60
|
||||
L_LUPI (privileged MSE) :done, 20, 60
|
||||
L_RKD (relational) :done, 20, 60
|
||||
L_seg (segmentation) :done, 20, 60
|
||||
L_CVD (disentanglement) :done, 20, 60
|
||||
|
||||
section Phases
|
||||
Warmup (L_task only) :milestone, 0, 10
|
||||
Ramp-up (+L_feat, L_CRD) :milestone, 10, 20
|
||||
Full (all 7 losses) :milestone, 20, 60
|
||||
```
|
||||
````
|
||||
|
||||
## Mermaid: Student Backbone Stages
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
graph LR
|
||||
IN["Input<br/>[B,3,256,256]"] --> S1["Stage 1<br/>[B,32,96,96]"]
|
||||
S1 --> S2["Stage 2<br/>[B,64,48,48]"]
|
||||
S2 --> BRIDGE["Conv1×1<br/>Bridge"]
|
||||
BRIDGE --> S3["Stage 3<br/>[B,128,24,24]"]
|
||||
S3 --> S4["Stage 4<br/>[B,256,12,12]"]
|
||||
S4 --> POOL["GGeM Pool"]
|
||||
POOL --> DESC["Descriptor<br/>[B,512]"]
|
||||
|
||||
S1 ~~~ NOTE1["SOFIA<br/>(DCN blocks)"]
|
||||
S3 ~~~ NOTE2["MambaVision<br/>(SSM+ViT)"]
|
||||
|
||||
style S1 fill:#fff3e0,stroke:#e65100
|
||||
style S2 fill:#fff3e0,stroke:#e65100
|
||||
style S3 fill:#e3f2fd,stroke:#1565c0
|
||||
style S4 fill:#e3f2fd,stroke:#1565c0
|
||||
style NOTE1 fill:#fff3e0,stroke:none,color:#e65100
|
||||
style NOTE2 fill:#e3f2fd,stroke:none,color:#1565c0
|
||||
```
|
||||
````
|
||||
|
||||
## Mermaid: Data Augmentation Pipeline
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
graph LR
|
||||
RAW[Raw RGB<br/>512×512] --> RESIZE[Resize<br/>256×256]
|
||||
RESIZE --> DA["Depth Anything V2<br/>Large (335M)"]
|
||||
RESIZE --> DSINE["DSINE<br/>(30M)"]
|
||||
RESIZE --> SEG["SegFormer-B5<br/>(85M)"]
|
||||
|
||||
DA --> D_OUT["depth.npy<br/>[1,256,256]"]
|
||||
DSINE --> N_OUT["normal.npy<br/>[3,256,256]"]
|
||||
SEG --> S_OUT["seg.npy<br/>[1,256,256]"]
|
||||
|
||||
RESIZE --> RGB["RGB<br/>[3,256,256]"]
|
||||
RGB --> CONCAT["concat_8ch.npy<br/>[8,256,256]"]
|
||||
D_OUT --> CONCAT
|
||||
N_OUT --> CONCAT
|
||||
S_OUT --> CONCAT
|
||||
|
||||
style DA fill:#e8f5e9,stroke:#2e7d32
|
||||
style DSINE fill:#fce4ec,stroke:#c62828
|
||||
style SEG fill:#e3f2fd,stroke:#1565c0
|
||||
```
|
||||
````
|
||||
Reference in New Issue
Block a user