framework
active
framework:k-fac-kronecker-factored-approximate-curvatureK-FAC (Kronecker-Factored Approximate Curvature)
Approximates FIM as G⊗A to give dataset-average curvature eigenbasis.
Neighborhood — ranked by edge-count
Papers (1)
paper
Thinkers (2)
thinker
- Roger GrosseintroducesUnpublished independent model diffing work.
- James MartensintroducesCo-originator of K-FAC.
Methods (1)
method
- K-FAC curvature-mass weight editingimplementsKeeps top eigenvector pairs by curvature mass to suppress memorization.
Concepts (2)
concept
- Curvature basis (K-FAC eigenbasis)implementsWeight components ordered high to low curvature.
- Fisher Information MatrixstudiesCurvature proxy approximated by K-FAC.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Basis for editing.
- K-FAC suppresses the broadest range of unseen memorized text without requiring labeled forget dataclaim0.719Practical advantage over BSN.
- Mathematical notation used throughout the paper to express operations that simultaneously act on position dimensions and vector dimensions of activations
- Weighted-sum summary of post-hoc reasoning for early answering and adding mistakes.
- K-FAC edit reduces OLMo-7B Dolma strict accuracy from 99.9% to 3.4% (BSN 6.0%, SVD 3.0%)finding0.687Table 1.
- Asymmetric measure of difference between two probability distributions.
- K-FAC eigenvector bands show sorted activation strength top10%>10-25%>25-50%>bottom50%, unlike SVDfinding0.675Supports curvature basis interpretability.
- Robustness.