method
active
method:path-expansion-methodPath Expansion Method
The core analytical technique of expanding transformer computations from layer-by-layer products into sums of end-to-end path terms for independent analysis
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- Prior Anthropic paper enabling circuit-level analysis of attention-only transformers; motivates current MLP decomposition
Concepts (1)
concept
- Residual Streamassociated_withProposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The mathematical trick of expanding a product of layer terms into a sum of end-to-end path terms, enabling independent analysis of each term
- Method by Goldowsky-Dill et al. 2023 for localizing model behavior via targeted activation interventions
- Neural mechanism for tracking location through accumulation of self-movement vectors; shown to play the role of position encodings in TEM.
- The path in activation space derived by fitting the representation manifold, used to steer along the geometric structure of internal representations.
- Feynman's quantum method where global behavior of light emerges from local behaviors with assigned probabilities; cited as example of global emerging from local
- An approach training agents to avoid unsafe pathways leading to deception, informed by Causal Influence Diagrams
- The number of distinct paths information can travel from point A to B in a transformer is C(m+n, n), quickly exceeding the number of atoms in the universe.
- Adding new states or parameters to the generative model if it increases model evidence, enabling concept learning.