finding
active
finding:on-acetylcholine-direct-attention-mlp-paths-each-favor-wrong-tokens-mrs-ely-ation-while-e-is-scored-highly-by-allOn ACETYLCHOLINE, direct, attention, MLP paths each favor wrong tokens (␣Mrs, ely, ATION) while 'E' is scored highly by all
No single path ranks E top.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Questions (1)
question
- Motivating question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Shows the passive vs. active divide is more important than the specific wording of instructions.
- Architectural rationale for why Style Modulation Heads are in attention layers rather than MLPs
- Largest IN Tokens→Logits weight votes 'utions', which never follows IN in training; Fisher effectiveness ~3 orders below the top weights.
- §5.5.
- Key limitation of the paper's approach; MLP layers make up 2/3 of standard transformer parameters
- Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
- Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
- Disentanglement evidence.