claim
active
claim:top-k-fac-eigenvectors-correspond-to-shared-generalizing-structure-while-low-curvature-components-underwrite-recitationTop K-FAC eigenvectors correspond to shared generalizing structure, while low-curvature components underwrite recitation
Basis for editing.
Source paper
extracted_from(2025) · Merullo, Jack · Vatsavaya, Srihita · Bushnaq, Lucius · Lewis, Owen
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- Table 2.
- K-FAC edit reduces OLMo-7B Dolma strict accuracy from 99.9% to 3.4% (BSN 6.0%, SVD 3.0%)associated_withTable 1.
- Regularization effect.
Claims (2)
claim
- Explains population vs instance curvature.
- Memorization is hard to pinpoint and likely highly distributed, though some curvature directions localize to early/late layersassociated_withConsistent with prior work.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Weight components ordered high to low curvature.
- Vision disentanglement.
- K-FAC eigenvector bands show sorted activation strength top10%>10-25%>25-50%>bottom50%, unlike SVDfinding0.780Supports curvature basis interpretability.
- Disentanglement evidence.
- Approximates FIM as G⊗A to give dataset-average curvature eigenbasis.
- If curvature measures memorization, eigenvector bands will activate differently on memorized vs clean datahypothesis0.753Section 4 hypothesis.
- Both empirical and theoretical phase diagrams.
- Distinct from internal emotion.