claim
active
claim:cross-layer-features-are-not-simply-passively-relayed-via-residual-connectionsCross-layer features are not simply passively relayed via residual connections
From drift findings.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- Features not merely relayed via residual connections.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Interpretation of eval loss result.
- Consistent but alternative explanations like amplification exist.
- Cross-layer transcoder features form a mix of local, global and intermediate features by decoder normfinding0.775Preliminary decoder norm analysis.
- Motivating example with 13 vs 3 features.
- Circuit simplification theory.
- The middle layer residual stream features are causally implicated in multi-step reasoning.claim0.754Features for Kobe Bryant, California, Lakers participate in computing the capital answer.
- Mask virtual weights during training targeting interference.
- Interpretation of pre/post MLP correlation.