claim
active
claim:current-saes-mix-global-subspace-and-local-tiling-solutions-in-a-fragmented-dilution-regimeCurrent SAEs mix global subspace and local tiling solutions in a fragmented dilution regime
Central empirical interpretation.
Source paper
extracted_from(2026) · Usha Bhalla · Thomas Fel · Can Rager · Sheridan Feucht +8
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (5)
finding
- Restricted-R² variance explained on Llama3.1-8B plateaus well beyond each manifold's ambient dimension across five SAE architecturesassociated_withsupportsSAEs do not achieve compact capture; they are in dilution.
- Top SAE features on the years manifold show localized smooth tuning curves with overlapping supportsupportsReminiscent of population coding.
- Specific tuning curve structure.
- Ambient-space shattering.
- Features act as local patch detectors.
Claims (2)
claim
- Framing in App. G.
- Geometry preserved but not organized usefully.
Questions (1)
question
- Central research question.
Methods (1)
method
- Restricted R² protocolsupportsGreedy atom selection measuring variance explained as function of support size.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Increasing TopK sparsity drives synthetic SAEs through shattering, capture, then dilution regimesfinding0.784Phase diagram on support size and receptive field spread.
- Additional synthetic example of pernicious divergence from balanced subspaces
- App. B.4.
- Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
- Theoretical framework answer.
- SAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.claim0.723Corollary 4.1.1.
- Claim about broader applicability of the scaling argument
- Core concern.