method
active
method:pre-post-mlp-crosscoderPre/post MLP crosscoder
Encoders read pre-MLP space; decoders reconstruct pre-MLP and MLP output to find newly computed features.
Neighborhood — ranked by edge-count
Methods (1)
method
- Sparse CrosscoderextendsSparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Pre/post MLP crosscoder shows a trimodal split into pre-only, shared and post-only featuresfinding0.867Relative decoder norm structure.
- Learn a shared feature space across two models; basis of the work.
- Extension of SAEs that jointly learns latents across representations from different models; proposed as alternative for extended fine-tuning
- A post-MLP feature for words indicating uniqueness has pre-MLP inputs firing on specific such wordsfinding0.753Abstract features built from specific instances.
- Each feature assigned a window of K layers to reduce training FLOPs.
- Architectural modification subtracting a learned bias from autoencoder inputs before encoding; initialized to geometric median of dataset; improves autoencoder performance
- MLP replacement with sparse features underlying the original attribution graphs.
- Basic crosscoder reading and reconstructing all layers jointly.