method
active
method:cross-layer-transcoder-cltCross-layer transcoder (CLT)
Replacement model architecture with 30M features replacing MLP neurons.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- SuperpositionaboutPhenomenon where models represent more features than dimensions via almost-orthogonal directions.
Methods (3)
method
- Gradient-based technique using SAE features to estimate causal effects on completions; used to corroborate NLA findings.
- Sparse CrosscoderextendsSparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
- CLT plus error nodes and original attention patterns, local to a prompt.
Claims (1)
claim
- Discussion.
Conceptual bridges
2-hop · via this method's ideasWhere ideas in this method connect to the rest of the corpus — the same concept, an analogy, or a restatement elsewhere.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- MLP replacement with sparse features underlying the original attribution graphs.
- Mask virtual weights during training targeting interference.
- Used to express MLP in monosemantic features, 4096 features with JumpReLU.
- Uses activations from one layer to predict the next; special case of crosscoders.
- Cross-layer transcoder features form a mix of local, global and intermediate features by decoder normfinding0.772Preliminary decoder norm analysis.
- Decomposition method for activations; VPD is compared against transcoders in sparsity-reconstruction tradeoff.
- Learn a shared feature space across two models; basis of the work.
- Decomposition approach mixing frozen base attention patterns.