finding
active
finding:topk-sae-per-block-r2-collapses-to-0-53-on-toy-manifold-superpositionTopK SAE per-block R² collapses to ≈0.53 on toy manifold superposition.
Baseline failure.
Source paper
extracted_from(2026) · Thomas Fel · Matthew Kowal · Mozes Jacobs · Dron Hazra +21
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Main theoretical claim, Prop. 1.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Toy superposition recovery.
- SMixAE as published reaches per-block R² ≈0.71; with signed code and block projection ≈0.85.finding0.756Repair of SMixAE.
- Justifies focusing on direction-based SAEs.
- Evidence in real models.
- Increasing TopK sparsity drives synthetic SAEs through shattering, capture, then dilution regimesfinding0.729Phase diagram on support size and receptive field spread.
- Authors' hedged interpretation.
- Strongest argument against pathology.
- Subspace capture on synthetic manifolds peaks near k=4 and degrades at lower and higher sparsityfinding0.723Sparsity sweet spot in synthetic benchmark (R^128, c=512).