claim
active
claim:smaller-models-weaker-concept-representations-explain-why-supervised-metrics-differ-on-pythia-160mSmaller models' weaker concept representations explain why supervised metrics differ on Pythia-160M
Interpretation of model-scale effects.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Model-scale dependence.
Questions (1)
question
- Open issue from Pythia vs Gemma.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Attributed to model anisotropy from saturation making hidden states harder to access
- Supervised metrics are limited by ground-truth availability and cover only a fraction of latentsclaim0.785Limitation.
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.776Selective pressure toward convergence via model capacity
- Surprising negative result for LDA despite being a supervised method
- Selective pressure toward convergence via task generality
- Supervised metrics discriminate.
- Earlier/less capable models exhibit a larger gap between think and don't think representation strengthfinding0.759Claude 3 models show a bigger difference than newer models like Opus 4.1.
- Core framing.