finding
active
finding:for-all-three-saes-1m-4m-34m-average-active-features-per-token-300-and-reconstruction-variance-explained-65For all three SAEs (1M, 4M, 34M), average active features per token <300, and reconstruction variance explained ≥65%.
Basic SAE performance metrics.
Source paper
extracted_from(2024) ·
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Table 1.
- The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
- Table 1.
- Most features dead in largest SAE, indicating room for improvement.
- Table 1.
- Fig 2 right.
- Turn-averaged features have highest embedding similarity 0.663 at k=3; per-token max lowest at 0.617finding0.784Consistent across k=1,2,3,5.
- Quantifies SAE reconstruction damage.