claim
active
claim:scaling-saes-on-discrete-features-learns-more-features-in-decreasing-order-of-frequencyScaling SAEs on discrete features learns more features in decreasing order of frequency
Discrete case.
Source paper
extracted_from(2025) · Michaud, Eric J. · Gorton, Liv · McGrath, Tom
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Warm-up derivation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Specific tuning curve structure.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
- Fig. 7d,e.
- Authors' experience-based claim.
- Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
- Core concern.
- Fig 2 right.
- A promising property for interpretability analysis off-distribution.