claim
active
claim:k-fac-suppresses-the-broadest-range-of-unseen-memorized-text-without-requiring-labeled-forget-dataK-FAC suppresses the broadest range of unseen memorized text without requiring labeled forget data
Practical advantage over BSN.
Source paper
extracted_from(2025) · Merullo, Jack · Vatsavaya, Srihita · Bushnaq, Lucius · Lewis, Owen
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (6)
finding
- Table 1.
- BSN nDCG@10 0.97 vs 0.91 for K-FAC and SVDcontradictsBSN preserves token ranking slightly better.
- Coherence comparison.
- Replication at smaller scale.
- Better transfer to unseen memorized data.
- Robustness.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- K-FAC eigenvector bands show sorted activation strength top10%>10-25%>25-50%>bottom50%, unlike SVDfinding0.753Supports curvature basis interpretability.
- Vision disentanglement.
- Basis for editing.
- §3.3.2.
- K-FAC edit on ViT-Base drops memorized train accuracy from 81.6% to 3.5%, GT accuracy to 66.5% (SVD 58.9%)finding0.727Table 2.
- Disentanglement evidence.
- Logic sometimes improves.
- Approximates FIM as G⊗A to give dataset-average curvature eigenbasis.