SAE Critique (Manifold Shattering)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• SAE features tend to shatter manifolds into many small and apparently-unrelated pieces, obscuring the overarching semantic structure.claim0.769
  Core critique of sparse autoencoders: they break the geometric structure of representations, making it harder to see the big picture.
• Manifold-level descriptions recover overarching semantic structure that SAE features miss.claim0.743
  Positive claim that geometric descriptions retain the conceptual coherence lost in atomized feature decompositions.
• SAE features can be grounded in clinical taxonomy (abnormality, age, sex, medication) to benchmark monosemanticity and entanglement.claim0.714
  Claim that feature grounding enables interpretability metrics.
• SAE-based mechanistic interpretability will be superseded by manifold-based analysis for understanding semantic concepts within 24 months.prediction0.713
• Some SAE concept steering interventions act as 'wrecking balls' that collapse global model performance rather than selectively modifying target concepts.claim0.703
  A critical failure mode identified in the paper demonstrating risk of naïve concept steering
• Sparse Autoencoders (SAE)method0.698
  Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
• Self-evaluated emotionality and textual evaluation of SAE features predict persistence in opposite directions.claim0.696
  Surprising finding that the two evaluation methods diverge in their relationship with persistence
• SAE features generalize to images despite training only on text, indicating out-of-distribution robustness.claim0.695
  A promising property for interpretability analysis off-distribution.