claim
active
claim:memorization-is-hard-to-pinpoint-and-likely-highly-distributed-though-some-curvature-directions-localize-to-early-late-layersMemorization is hard to pinpoint and likely highly distributed, though some curvature directions localize to early/late layers
Consistent with prior work.
Source paper
extracted_from(2025) · Merullo, Jack · Vatsavaya, Srihita · Bushnaq, Lucius · Lewis, Owen
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Top K-FAC eigenvectors correspond to shared generalizing structure, while low-curvature components underwrite recitationassociated_withBasis for editing.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Section 6.
- Core novel interpretation.
- If curvature measures memorization, eigenvector bands will activate differently on memorized vs clean datahypothesis0.800Section 4 hypothesis.
- Section 4.
- Layer-wise trajectories show early enrichment, mid-layer alignment, and late re-clustering.claim0.770Qualitative geometry pattern.
- Proposes that nonlinear geometric structure is superior to linear feature spaces for capturing semantic content.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.769Explanation for why dictionary learning can recover many more features than dimensions.
- The same block construction predicts a richer contour code (curvature plus position) in deeper layers.hypothesis0.768Forward prediction analogous to V4.