finding
active
finding:0-46-asymptotic-vs-0-34-compute-optimal-in-practice-so-asymptotic-loss-of-larger-models-exceeds-that-of-smaller-onesα>γ (α≈0.46 asymptotic vs γ=0.34 compute-optimal) in practice, so asymptotic loss of larger models exceeds that of smaller ones
Basis of phenomenological argument.
Source paper
extracted_from(2026) · Jing Huang · Daniel Wurgaft · Rachit Bansal · Laura Ruis +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Power-law scaling already implies a larger model can learn a part of the distribution a smaller model fails to learn even with infinite dataassociated_withsupportsCore phenomenological argument.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig. 22.
- Quantitative threshold used for accepting reduced models; linked to Bayes factor of ~20
- Extrapolation of scaling predictive models to AGI.
- Fig. 23.
- Implication of PRH for AI fairness and bias
- what α value provides the optimal balance between steering accuracy and response stability for different CV methods?question0.758Practical hyperparameter question motivating the injection strength sweep experiment
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific featuresclaim0.753All experiments used L1-of-norms.
- Interpretation of effectiveness tail result.