finding
active
finding:saes-trained-on-pretraining-data-lack-good-latents-for-refusal-while-saes-trained-on-chat-formatted-data-have-themSAEs trained on pretraining data lack good latents for refusal, while SAEs trained on chat-formatted data have them
SAE latents depend on training data.
Source paper
extracted_from(2025) · Lee Sharkey · Bilal I. Chughtai · Joshua Batson · Jack W. Lindsey +25
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Core assessment of SDL.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Feature cluster examples from the UMAP.
- Prediction following diversity-undertraining logic.
- Chat-trained SAEs find behaviourally relevant features because many causal features occur on chat-specific special tokenshypothesis0.803Explanation for LMSYS superiority.
- App. B.
- Comparison against LMSYS training.
- Authors' experience-based claim.
- Refusal is not a single linear classifier.
- Consistent with refusal cone work.