method
active
method:single-layer-transcoderSingle-layer transcoder
Used to express MLP in monosemantic features, 4096 features with JumpReLU.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.
Methods (2)
method
- Transcodersrelated_toDecomposition method for activations; VPD is compared against transcoders in sparsity-reconstruction tradeoff.
- Transcoderrelated_toUses activations from one layer to predict the next; special case of crosscoders.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- MLP replacement with sparse features underlying the original attribution graphs.
- Mask virtual weights during training targeting interference.
- Replacement model architecture with 30M features replacing MLP neurons.
- Decomposition approach mixing frozen base attention patterns.
- Second category of SAE/transcoder concerns, distinct from faithfulness.
- Discussion.
- Learn a shared feature space across two models; basis of the work.
- A transformer with no attention layers; shown to model bigram statistics via T = W_U W_E