claim
active
claim:rank-1-ov-decompositions-split-higher-rank-ov-behaviors-like-succession-or-copying-induction-across-many-featuresRank-1 OV decompositions split higher-rank OV behaviors like succession or copying induction across many features
Limitation of existing decompositions.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- Decomposition approach with rank-1 OV directions.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- What matrix decomposition or dimensionality reduction best summarizes the enormous low-rank OV and QK matrices?question0.767Open methodological question about converting the 50k x 50k expanded matrices into human-graspable summaries
- Prediction for future decomposition.
- Rank-1 tasks (K=32, β=2) show empirical transition width matching N_crit(k)=k staircase almost perfectlyfinding0.750Sharp test of utility ordering.
- Which trait pairs combine cleanly, which produce a single dominant trait, and which collapse both?question0.746The open question motivating the pairwise composition study
- Constraint in VPD where each parameter subcomponent is constrained to be a rank-one matrix for simplicity.
- Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
- Empirical finding from Tulu-70B ablation study across post-training stages