thinker
active
thinker:alexander-modell

Alexander Modell

Formal notion of feature manifolds; cosine similarity encodes geometry.

Authored
1
Introduces
0
Studies
0
Affiliations
0
Cited by
4

Authored papers (1)

  • Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token length are each embedded with high curvature in small residual-stream subspaces, and the model computes with them through linear geometric operations. Character count lives on a curve in a 6-dimensional subspace (95% of variance of 150 mean activations at layer 2), parametrized by 10 crosscoder features from a 10M-feature Weakly Causal Crosscoder. Attention 'boundary heads' twist the count manifold through QK so count i aligns with line width k=i+ε, raising probe cosine similarity from ~0.25 in the residual stream to ≈1 in QK space. Several heads with different offsets tile 'characters remaining'. Counts for characters remaining and next-token length sit in near-orthogonal subspaces, making the break decision linearly separable (AUC 0.91). The curvature is built by many attention heads, with 5 layer 0 heads reaching R² 0.93 and 11 heads in layers 0-1 reaching 0.97. The paper pairs attribution graphs and feature dictionaries with explicit manifold parametrization, and validates with subspace ablations, mean-activation interventions and '@@' visual illusions. Ringing is shown to be optimal under a capacity-versus-resolution tradeoff. Dictionary features alone impose a complexity tax that geometry reduces, and natural pretraining tasks are good targets for mechanistic biology, though parametrizing manifolds remains manual.

More papers — OpenAlex / S2

Co-authors (9)

Recent mentions (1)