claim
active
claim:qualitatively-new-representations-emerge-in-middle-layers-as-model-scale-increasesQualitatively new representations emerge in middle layers as model scale increases
Interpretation of NMF result.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- Scale-dependent middle-layer representations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The observation that larger LLMs develop more general, abstract linear representations (e.g., truth across diverse topics) compared to smaller models
- Interpretation of weaker PCA separation and lower ASR in smaller models
- Interpretive claim connecting scale to abstraction level in LLM representations
- Theoretical interpretation of antipodal alignment and misalignment phenomena in PCA visualizations
- The causal hypothesis motivating the use of causality (intervention) as the lens connecting representation and behavior geometry.
- Core methodological hypothesis enabling the application of IIT to LLM representation sequences.
- How do representations differ or converge between architectures, tasks, and modalities?question0.777Broader research question MAS is positioned to address, citing multiple recent works.
- Interpretation of the layer-by-layer PCA visualizations showing linear structure emerging in early-middle layers