claim
active
claim:seemingly-smooth-loss-decreases-in-larger-models-are-composed-of-many-small-feature-jumpsSeemingly smooth loss decreases in larger models are composed of many small feature jumps
Suspicion from energy level jumps.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Findings (1)
finding
- Features jump between dimensionalities during training with sudden loss drops in digon modelssupportsLearning dynamics.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig. 22.
- Fig. 23.
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.770Selective pressure toward convergence via model capacity
- Baseline.
- Fig. 7d,e.
- Core framing.
- Hypothesis box in §3.2.
- Extrapolation of scaling predictive models to AGI.