finding
active
finding:saes-trained-on-pretraining-data-lack-good-latents-for-refusal-while-saes-trained-on-chat-formatted-data-have-them

SAEs trained on pretraining data lack good latents for refusal, while SAEs trained on chat-formatted data have them

SAE latents depend on training data.

Source paper

extracted_from
Open Problems in Mechanistic Interpretability
(2025) · Lee Sharkey · Bilal I. Chughtai · Joshua Batson · Jack W. Lindsey +25

Neighborhood — ranked by edge-count