claim
active
claim:number-of-attention-head-paths-between-transcoder-features-grows-exponentially-with-layer-separationNumber of attention head paths between transcoder features grows exponentially with layer separation
Why checkpointing is needed.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Strategy forcing edges to be mediated only by head paths of length 1 to avoid exponential path growth.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Result from term importance analysis breaking down loss contribution by layer
- Striking mechanistic finding that injection creates universally detectable perturbation in residual stream immediately downstream
- Empirical observation from examining expanded OV/QK matrices; approximately 10 out of 12 heads show significant copying
- Concrete example from examining expanded QK/OV matrices showing how specific programming language structure is encoded in attention weights
- Identification of algorithms implemented in attention layers, distributed across attention headsfinding0.784VPD successfully recovered interpretable attention algorithms (previous-token behavior, syntax-boundary routing) in weight space without requiring manual decomposition across heads.
- Prediction for future decomposition.
- Suspected but no confirmed example in real LLM.
- Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.