claim
active
claim:crosscoders-let-attribution-hop-back-to-the-layer-where-a-feature-was-computed-uncovering-circuits-much-shallower-than-model-depthCrosscoders let attribution hop back to the layer where a feature was computed, uncovering circuits much shallower than model depth
Circuit simplification theory.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Claims (1)
claim
- Motivating example with 13 vs 3 features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Motivating problem.
- Decoder cost dominates.
- Crosscoders are about 2x less FLOP-efficient than per-layer SAEs at equal eval loss at large computefinding0.798Cost of identifying cross-layer structure.
- Motivation for DFC.
- Due to SAE training stochasticity.
- Indicates redundant structure across layers.
- Conceptual difference.
- Crosscoders give a dramatically better whole-model (MSE, L0) tradeoff than summed per-layer SAEsfinding0.784Consolidating shared structure gives concise decomposition.