finding
active
finding:turn-averaged-and-per-token-saes-both-surface-persona-relevant-features-evil-hallucination-sycophancy-poetic-etc-via-contrastive-rollouts-with-noiseTurn-averaged and per-token SAEs both surface persona-relevant features (evil, hallucination, sycophancy, poetic, etc.) via contrastive rollouts, with noise
Appendix I.
Source paper
extracted_from(2026) · Kevin Der · Harish Kamath · Thompson, Ben
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Turn-averaged features will outperform per-token features on whole-turn tasks such as summarizing multi-turn exchanges or judging persona/safety traitsassociated_withsupportsPrediction from coverage results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
- Abstract.
- Table 7.
- Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
- Example B.
- Author's hypothesis explaining why persona vectors extracted from exhibited-trait activations generalize to causal influence
- Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.quote0.794Abstract thesis.
- Supported by TruthfulQA generalization in Experiment 2: same feature directions gate factual accuracy across 29 independent categories