Using attention sinks to identify and evaluate dormant heads in pretrained llms

Similar preprints — Semantic Scholar

Cited by (2)

• A Mechanistic Analysis of Looped Reasoning Language Models

  Looped reasoning language models converge to cyclic fixed-point behavior in latent space: each layer in a recurrent block approaches a distinct fixed point, so the block traces a consistent cyclic tra

• Model Alignment Search

  Model Alignment Search (MAS) establishes bidirectional causal similarity between neural networks by learning a per-model orthogonal rotation matrix that isolates behaviorally relevant subspaces and us