thinker:emmanuel-ameisenEmmanuel Ameisen
Co-author; enabled feature-level attributions, multi-step inference analysis.
Authored papers (4)
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token length are each embedded with high curvature in small residual-stream subspaces, and the model computes with them through linear geometric operations. Character count lives on a curve in a 6-dimensional subspace (95% of variance of 150 mean activations at layer 2), parametrized by 10 crosscoder features from a 10M-feature Weakly Causal Crosscoder. Attention 'boundary heads' twist the count manifold through QK so count i aligns with line width k=i+ε, raising probe cosine similarity from ~0.25 in the residual stream to ≈1 in QK space. Several heads with different offsets tile 'characters remaining'. Counts for characters remaining and next-token length sit in near-orthogonal subspaces, making the break decision linearly separable (AUC 0.91). The curvature is built by many attention heads, with 5 layer 0 heads reaching R² 0.93 and 11 heads in layers 0-1 reaching 0.97. The paper pairs attribution graphs and feature dictionaries with explicit manifold parametrization, and validates with subspace ablations, mean-activation interventions and '@@' visual illusions. Ringing is shown to be optimal under a capacity-versus-resolution tradeoff. Dictionary features alone impose a complexity tax that geometry reduces, and natural pretraining tasks are good targets for mechanistic biology, though parametrizing manifolds remains manual.
A sparse, privileged subset of residual-stream directions in Claude models meets five functional criteria of a global workspace, and it is found by asking which representations the model is poised to verbalize. The Jacobian lens (J-lens) averages the first-order effect of each layer's activation on future token logits over 1,000 pretraining-like prompts, which corrects the logit lens's coordinate mismatch and yields the J-space, a sparse nonnegative subframe of about k≤25 token vectors. On Sonnet 4.5 the J-space carries at most 10% of activation variance, yet it governs verbal report, directed modulation, internal reasoning and flexible generalization. Swapping lens coordinates redirects two-hop answers in 54% (Haiku 4.5) to 70% (Sonnet 4.5, Opus 4.5) of trials. The J-space component of a concept probe flips answers in 61% of trials against 28% for the non-J remainder, and clamping the J-space drops that remainder to 6%. Ablating the top-10 J-lens vectors leaves MMLU, SQuAD and CoLA intact but degrades multi-hop reasoning, translation and sonnet writing to below unablated Haiku 4.5. The workspace occupies layers ~L38–92, and a 1% subset of attention heads broadcasts it. Applications follow: ablating eval-awareness tokens raises blackmail from 0/180 to 13/180 rollouts, and counterfactual reflection training cuts Haiku 4.5's fabrication dishonesty from 0.25 to 0.07 through J-space content. LLMs thus contain a readable, editable functional analogue of conscious access, with its limits set by single-token vocabulary and no recurrence.
More papers — OpenAlex / S2
Affiliations (1)
- Anthropic(institute)
Co-authors (12)
- Joshua Batson7 shared
- Adam Pearce6 shared
- Chris Olah6 shared
- Isaac Kauvar5 shared
- Wes Gurnee5 shared
- Julius Tarng4 shared
- Jack Lindsey3 shared
- Alexander Modell2 shared
- Eric Michaud2 shared
- Gorton2 shared
- Adam Jermyn1 shared
- Adly Templeton1 shared
Their work is cited by (12)
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior2× refs
- Persona Features Control Emergent Misalignment2× refs
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds2× refs
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution2× refs
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts2× refs
- Do Sparse Autoencoders Capture Concept Manifolds?2× refs
- Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders1× refs
- Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering1× refs
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire1× refs
- When Models Manipulate Manifolds: The Geometry of a Counting Task1× refs
- Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought1× refs
- Adversarial Examples Are Not Bugs, They Are Superposition1× refs
Other inbound relations (5)
- mentionsReplicating Circuit Tracing for a Simple Known Mechanism - Goodfire(paper)
- mentionsTurn-Averaged SAEs for Feature Discovery and Long-Context Attribution(paper)
- mentionsUnder the Hood of a Reasoning Model - Goodfire(paper)
- mentionsVerbalizable Representations Form a Global Workspace in Language Models(paper)
- mentionsWhen Models Manipulate Manifolds: The Geometry of a Counting Task(paper)
Recent mentions (6)
- papers-typedhazra-hood-reasoning-0b9305.md
- papers-typedreplicating-circuit-2e7989.md
- papers-typedmanipulate-manifolds-b500bc.md
- papers-typedder-2026-turn-averaged.md
- papers-typedauthors-verbalizable-representations.md
- papersscaling.md