thinker:gortonGorton
Observed ripples in Inception v1 curve detectors.
Authored papers (1)
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token length are each embedded with high curvature in small residual-stream subspaces, and the model computes with them through linear geometric operations. Character count lives on a curve in a 6-dimensional subspace (95% of variance of 150 mean activations at layer 2), parametrized by 10 crosscoder features from a 10M-feature Weakly Causal Crosscoder. Attention 'boundary heads' twist the count manifold through QK so count i aligns with line width k=i+ε, raising probe cosine similarity from ~0.25 in the residual stream to ≈1 in QK space. Several heads with different offsets tile 'characters remaining'. Counts for characters remaining and next-token length sit in near-orthogonal subspaces, making the break decision linearly separable (AUC 0.91). The curvature is built by many attention heads, with 5 layer 0 heads reaching R² 0.93 and 11 heads in layers 0-1 reaching 0.97. The paper pairs attribution graphs and feature dictionaries with explicit manifold parametrization, and validates with subspace ablations, mean-activation interventions and '@@' visual illusions. Ringing is shown to be optimal under a capacity-versus-resolution tradeoff. Dictionary features alone impose a complexity tax that geometry reduces, and natural pretraining tasks are good targets for mechanistic biology, though parametrizing manifolds remains manual.
More papers — OpenAlex / S2
Co-authors (9)
- Adam Pearce2 shared
- Chris Olah2 shared
- Emmanuel Ameisen2 shared
- Isaac Kauvar2 shared
- Joshua Batson2 shared
- Julius Tarng2 shared
- Wes Gurnee2 shared
- Alexander Modell1 shared
- Eric Michaud1 shared
Their work is cited by (4)
- Do Sparse Autoencoders Capture Concept Manifolds?1× refs
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior1× refs
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds1× refs
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts1× refs
Other inbound relations (1)
Recent mentions (1)
- papers-typedmanipulate-manifolds-b500bc.md