finding
active
finding:weakly-causal-crosscoder-flop-efficiency-lies-between-per-layer-saes-and-acausal-crosscoders-lagging-global-crosscoders-by-3-4x-in-dictionary-sizeWeakly causal crosscoder FLOP efficiency lies between per-layer SAEs and acausal crosscoders, lagging global crosscoders by 3-4x in dictionary size
Naive implementation results.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Hypotheses (1)
hypothesis
- Decoder cost dominates.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Crosscoders are about 2x less FLOP-efficient than per-layer SAEs at equal eval loss at large computefinding0.872Cost of identifying cross-layer structure.
- Indicates redundant structure across layers.
- Source of the features used in attributions.
- Dictionary used for all main-text features.
- Motivating example with 13 vs 3 features.
- Motivation for DFC.
- Circuit simplification theory.
- Decoder norm profile across 18 layers.