thinker:openalex-A5124153233Sonia Joseph
Authored papers (1)
A sparse autoencoder trained on layer-19 residual-stream activations of Llama-3-8B-Instruct, using LMSYS-Chat-1M, yields features that can be labeled automatically, ranked by causal relevance, and used to steer chat behavior, and training-data choice decides whether those features are usable for intervention. Goodfire built a ReLU SAE with MSE plus L1 loss, following the Anthropic April 2024 update settings. Its explicit rationale for avoiding TopK and JumpReLU was concern about high-frequency features. Claude labels each live feature from activation samples across the full activation range. Labels are scored contrastively against distractors and against a feature's ten nearest embedding neighbours. The attribution instrument is a gradient of the predicted-token logit minus the mean logit with respect to SAE features, summed over token positions. Interventions are scaled between −h_max and +h_max, and the SAE error term ε is held fixed. Chat-data SAEs found behaviourally relevant features, often on chat special tokens. Non-chat FineWeb data and diverse mixes gave more dead and low-frequency features and poorer intervention candidates. Goodfire conjectures that features become interpretable before they become intervenable. Negative interventions often need several features set below zero, which is consistent with echo features and self-repair. Some features resist steering, possibly because of cross-layer superposition. The implication is that SAE quality for steering depends on where activations were harvested and on causal validation. Interpretability alone does not establish it. Autointerp must also move beyond token-level labels.
More papers — OpenAlex / S2
Co-authors (6)
- Caden Juang1 shared
- Danilo Bzdok1 shared
- Jack Stanley1 shared
- Lee Sharkey1 shared
- Luca Scimeca1 shared
- Praneet Suresh1 shared