pikaliov 8f8cbb14dd Diagnostic baseline v2: also disable MoCo queue ...

The mutex-only run still collapsed at epoch 1 — same pattern as the DSS
run. Val loss locks to log(8) ≈ 2.08 (uniform over the in-batch sat),
train loss grows monotonically (4.80 → 5.56), train R@1 drops 7.79% →
0.34%. Mode collapse, not sampler-induced.

The smoking gun is the queue: WeightedInfoNCELoss (the OLD-run loss)
silently ignored `queue_negatives`, so the OLD-run effective task was
in-batch-only contrast against 8 negatives. Switching to InfoNCELoss
made the queue active — 4096 stale embeddings without a momentum encoder
to keep them consistent with the live model. With the trimmed adapter
surface (MONA in last 12/24 blocks → 3.5M trainable), the model can't
reconcile fresh representations against stale negatives and collapses.

Disable the queue entirely (`neg_bank_size = 0`). Matches OLD's effective
setup — same 8 in-batch negatives, but with the new SymmetricInfoNCE +
mutex sampler + tau clamp 0.1 + per-sample mask + full-gallery eval.

Output → `out/gtauav/baseline_inbatch` (separate from the failed mutex run).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-04-25 05:57:14 +03:00

1.4 KiB

Raw Permalink Blame History

View Raw