finding
active
finding:sae-advantage-over-pca-on-sparse-probing-grows-substantially-from-k-1-to-k-5SAE advantage over PCA on sparse probing grows substantially from K=1 to K=5
Multiple latents capture concepts.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Probing limited.
- Supervised metrics discriminate.
- RM features at rank 181, outside top 100.
- Characterizes the differential sensitivity to injection strength between SAE and CAA methods
- Exception to ReLU underperformance.
- Table 4.
- The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
- Authors' experience-based claim.