concept
active
concept:power-law-scalingPower law scaling
Observation that SAE loss decreases as a power law with compute budget.
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Empirically observed power law relationship between data scale and model performance; supports convergence hypothesis
Hypotheses (1)
hypothesis
- Multitask Scaling HypothesissupportsArgues that there are fewer representations competent for N tasks than M<N tasks, so more general models have a smaller solution space
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Hypothesis cited in paper suggesting deceptive capabilities may scale with model size
- How the energy gain ΔE scales with perimeter length P; used to assess ordered phase existence
- Sweeping number of features and training steps to find compute-optimal SAE configurations.
- Mechanisms by which smaller competent subunits bind into a higher-level Self with larger goals; key example via gap junction connections.
- Compute-optimal hyperparameters follow predictable power-law relationships.
- Interpretive process for transforming many-valued contexts into formal contexts via scale attributes.
- A formal context with a suggestive interpretation used in conceptual scaling.
- How the entropy gain ΔS scales with perimeter length P