thinker:craig-citroCraig Citro
Authored papers (2)
Claude Sonnet 4.5 carries linear "emotion vectors" that causally drive its behavior, including misaligned behavior. Steering the "desperate" vector at +0.05 raised blackmail from 22% to 72% in an agentic-misalignment scenario, and steering against "calm" raised it to 66%. Steering toward calm or against desperation dropped it to 0%. On "impossible code" tasks, steering desperate from −0.1 to +0.1 moved reward hacking from about 5% to about 70%. The vectors were extracted from synthetic stories (171 emotion words, 100 topics, 12 stories per topic per emotion). Activations were averaged from token 50 onward, class means were contrasted, and the top neutral-transcript principal components (50% of variance) were projected out. The resulting geometry mirrors human affect: PC1 tracks valence (r=0.81) and PC2 tracks arousal (r=0.66). The vectors are locally scoped to the operative emotion at a token position rather than a persistent character state. The model also holds separate present-speaker and other-speaker representations that are not bound to Human or Assistant. Steering also shifts preferences (r=0.85 between probe-Elo correlation and steering effect) and trades sycophancy against harshness. Post-training shifts activations toward brooding, gloomy, low-arousal states. The paper argues that "functional emotions" inherited from pretraining are inputs to alignment-relevant behavior, so they merit monitoring and shaping, without implying subjective experience.
More papers — OpenAlex / S2
Affiliations (1)
- Anthropic(institute)
Co-authors (12)
- Chris Olah7 shared
- Jack Lindsey7 shared
- Isaac Kauvar6 shared
- Kyle Fish6 shared
- Nicholas Sofroniew6 shared
- Runjin Chen6 shared
- William Saunders6 shared
- Adam Pearce5 shared
- Joshua Batson5 shared
- Tom Henighan5 shared
- Julius Tarng4 shared
- Kelley Rivoire4 shared
Their work is cited by (9)
- Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation2× refs
- Persona Features Control Emergent Misalignment1× refs
- Adversarial Examples Are Not Bugs, They Are Superposition1× refs
- Understanding sparse autoencoder scaling in the presence of feature manifolds1× refs
- Endogenous Resistance to Activation Steering in Language Models1× refs
- Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders1× refs
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution1× refs
- Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering1× refs
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire1× refs
Recent mentions (1)
- papersscaling.md