claim
active
claim:matryoshka-s-hierarchical-structure-explains-its-positive-scaling-on-disentanglement-metricsMatryoshka's hierarchical structure explains its positive scaling on disentanglement metrics
Mechanistic interpretation.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- SCR at fixed intervention budget decreases with width for most architectures while Matryoshka improves on Gemma-2-2Bassociated_withsupportsInverse scaling on SCR.
- Inverse scaling.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Explains inverse scaling.
- Metrics worsen with dictionary size for most architectures.
- A combination of crosscoders and flexible decoding (Matryoshka, iterative codebook) could resolve the steering-classification tensionhypothesis0.742Proposed remedy.
- Model-scale dependence.
- Finding (2).
- Skyline.
- Rank-1 tasks (K=32, β=2) show empirical transition width matching N_crit(k)=k staircase almost perfectlyfinding0.737Sharp test of utility ordering.