question
active
question:as-we-make-a-model-wider-do-we-just-get-more-features-or-the-same-features-packed-less-denselyAs we make a model wider, do we just get more features, or the same features packed less densely?
Fresh-traction question.
Source paper
extracted_fromRelated by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.800Selective pressure toward convergence via model capacity
- Width buys features in order of utility, so only larger models learn infrequent or complex tasksclaim0.790Interpretation of Theorem 3.
- Can we transform a model's features into an inexpensive, dense signal for learning open-ended tasks?question0.788Motivating question.
- Explains why larger models (gemma-3-12b-it, Qwen3-30B) show multiple transition layers rather than a single one
- Inverse scaling interpretation.
- Central interpretation.
- Primary unexpected empirical observation.
- Features may not be strictly one-dimensional objects; higher-dimensional feature manifolds may exist in model representationshypothesis0.772Extension of superposition hypothesis to account for continuous families of features