finding
active
finding:relu-saes-show-comparable-performance-in-the-unlearning-evaluationReLU SAEs show comparable performance in the unlearning evaluation
Unlearning not discriminative.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Exception to ReLU underperformance.
- Discrete case.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
- SAE latents depend on training data.
- ReLU underperforms.
- App. B.
- Assistant-only SAEs reconstruct user turns 1.5–3× worse; both-roles training cuts user-turn nMSE 53–63%finding0.748Appendix G.
- Prediction following diversity-undertraining logic.