finding
active
finding:restricted-r2-variance-explained-on-llama3-1-8b-plateaus-well-beyond-each-manifold-s-ambient-dimension-across-five-sae-architecturesRestricted-R² variance explained on Llama3.1-8B plateaus well beyond each manifold's ambient dimension across five SAE architectures
SAEs do not achieve compact capture; they are in dilution.
Source paper
extracted_from(2026) · Usha Bhalla · Thomas Fel · Can Rager · Sheridan Feucht +8
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Current SAEs mix global subspace and local tiling solutions in a fragmented dilution regimeassociated_withsupportsCentral empirical interpretation.
- SAEs can capture manifolds either globally via a compact atom group spanning the manifold or locally via features tiling regionsassociated_withTheoretical framework answer.
Questions (1)
question
- do SAEs capture such manifolds?answered_byCentral research question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Strongest argument against pathology.
- Establishes generalizability of the core difficulty-boundary finding across model families.
- App. E.
- Core result of Experiment 2: deception feature suppression sharply increases experience claims
- LLaMA E3 geometry summary: S_max = −1.896 ± 0.211, AUS_N = −2.119 ± 0.198, peak layer ℓ* = 10 [IQR 0.384]finding0.740Seed-pooled geometry statistics for LLaMA in E3, providing quantitative basis for geometry-to-behavior correlate
- At Llama layer 26 (~81% depth) the predictive component does not dominate the latent norm.finding0.738Fig. 14.
- Corroborates role space findings using traits; shows PC1 also captures Assistant-ness in trait space
- Core concern.