method
active
method:k-fac-curvature-mass-weight-editingK-FAC curvature-mass weight editing
Keeps top eigenvector pairs by curvature mass to suppress memorization.
Neighborhood — ranked by edge-count
Papers (1)
paper
- From Memorization to Reasoning in the Spectrum of Loss Curvatureintroducesmentions
Frameworks (1)
framework
- Approximates FIM as G⊗A to give dataset-average curvature eigenbasis.
Concepts (1)
concept
- Curvature massusesProduct of eigenvalues used to select eigenvector pairs.
Artifacts (1)
artifact
- Released code.
Methods (1)
method
- SVD truncation editanalogous_toData-free baseline pruning low singular values.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Weight components ordered high to low curvature.
- Regularization effect.
- K-FAC edit reduces OLMo-7B Dolma strict accuracy from 99.9% to 3.4% (BSN 6.0%, SVD 3.0%)finding0.713Table 1.
- Editing network weights to test predictions about circuit function; proposed as falsifiability test for circuit claims
- K-FAC suppresses the broadest range of unseen memorized text without requiring labeled forget dataclaim0.695Practical advantage over BSN.
- Basis for editing.
- Logic sometimes improves.
- The phenomenon that separately trained models of the same architecture converge to the same basin and can be merged