finding
active
finding:wide-toy-models-show-lower-gradient-cosine-similarity-between-frequent-and-rare-gradients-so-frequent-updates-do-not-overwrite-rare-featuresWide toy models show lower gradient cosine similarity between frequent and rare gradients, so frequent updates do not overwrite rare features
Fig. 4 caption.
Source paper
extracted_from(2026) · Jing Huang · Daniel Wurgaft · Rachit Bansal · Laura Ruis +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Hypothesis box in §3.2.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Mechanism from Theorem 4.
- Larger rotations between shared features skew decoder cosine similarities downward in the toy modelfinding0.811Supports 'same features, used differently' interpretation.
- Validates toy model.
- Supports the co-activation explanation of symmetry.
- Figure 4.
- Fig. 4.
- Symmetry is not explained by the base toy model.
- Third empirical pattern.