paper:sparse-crosscoders-994b59Sparse Crosscoders for Cross-Layer Features and Model Diffing
Methods (13)
- Acausal crosscoderBasic crosscoder reading and reconstructing all layers jointly.
- Cross-layer transcoder (CLT)Replacement model architecture with 30M features replacing MLP neurons.
- L1-of-norms regularizationSparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
- L2-of-norms regularizationAlternative penalty that spreads features across layers and finds only shared features.
- Locally masked convolutional crosscoderEach feature assigned a window of K layers to reduce training FLOPs.
- meta-SNEAlgorithm canonicalizing network representations by distances.
- Model StitchingTechnique to measure representational compatibility by integrating intermediate representations of one model into another
- Nonnegative matrix factorization of decoder normsApplied to 30-dim (layer, model) norm vectors to find shared structure across scale.
- Pre/post MLP crosscoderEncoders read pre-MLP space; decoders reconstruct pre-MLP and MLP output to find newly computed features.
- Sparse CrosscoderSparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
- SVCCAAlternative representational alignment metric compared against mutual k-NN in experiments
- TranscoderUses activations from one layer to predict the next; special case of crosscoders.
- Weakly causal crosscoder (10M features)Source of the features used in attributions.
Frameworks (3)
- CrosscodersLearn a shared feature space across two models; basis of the work.
- Sparse AutoencoderInterpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
- Superposition HypothesisCore theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
Datasets (2)
- 18-layer model residual stream activationsActivations used to compare crosscoder vs per-layer SAEs.
- Claude 3 Sonnet and base modelResidual stream activations from the middle layer used for finetuning diff.
Findings (17)
- Crosscoder on Claude 3 Sonnet and base middle layer yields three groups: base-specific, finetuned-specific, shared, with 4-5 thousand model-specific features each of 1M
Decomposition into shared and model-specific features.
- Shared features' decoder vectors are almost always highly aligned across base and finetuned model, with a few thousand showing low or negative correlation
Same concept and function in both models mostly.
- Weakly causal crosscoder FLOP efficiency lies between per-layer SAEs and acausal crosscoders, lagging global crosscoders by 3-4x in dictionary size
Naive implementation results.
- NMF with four components on three models x ten layers: early layers, later layers, and middle layers of smallest vs larger two models
Scale-dependent middle-layer representations.
- Finetuned-specific features include a refusal feature, a code review feature, and a feature for personal questions to the Assistant
Cherrypicked interpretable examples.
- Most crosscoder features peak in a particular layer and decay gradually, many with substantial norm across most or all layers
Decoder norm profile across 18 layers.
- Stable features show positive but high-variance, low-in-absolute-terms correlation between pre- and post-MLP directions
May explain drift: MLPs relay features along different axes.
- Feature decoder directions are more stable than chance but drift substantially across layers even where norm is strong
Features not merely relayed via residual connections.
- Eval loss interpolates fairly linearly as convolutional window K varies from 1 to n_layers, with no inflection point
Locally masked crosscoder equals smaller FLOP-matched global crosscoder.
- Crosscoders substantially outperform per-layer SAEs on eval loss at fixed total feature count on 18-layer model
Indicates redundant structure across layers.
Claims (13)
- Per-layer SAEs produce duplicate persistent features that make circuits seem more complex than needed; crosscoders can radically simplify circuits
Motivating example with 13 vs 3 features.
- Crosscoder causal graph may differ from the model's actual mechanism even if activations are isomorphic, so interventional experiments need care
Downside of abstraction.
- Yun et al. define sameness by direction across layers while crosscoders define it by activating on the same data points, allowing feature drift
Conceptual difference.
- Crosscoders let attribution hop back to the layer where a feature was computed, uncovering circuits much shallower than model depth
Circuit simplification theory.
- The crosscoder diff approach is likely preferable when finetuning uses a greater fraction of compute relative to pretraining
Compared with base SAE transfer.
- There is significant redundant, linearly correlated structure across layers interpreted as cross-layer features
Interpretation of eval loss result.
- Shared features with low or negative alignment likely reflect finetuned model using a base concept in a new way
Suspicion not investigated in depth.
- Crosscoders bake in clustering of features across layers at training time that is hard to do post hoc on SAEs
Due to SAE training stochasticity.
- MLP layers relay many features without nonlinear modification but along a different axis, explaining drift
Interpretation of pre/post MLP correlation.
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific features
All experiments used L1-of-norms.
Hypotheses (3)
- Adversarially robust models have different interpretability because adversarial training affects superposition
Could be tested by crosscoder comparisons.
- Gradual feature formation across layers is evidence of cross-layer superposition
Consistent but alternative explanations like amplification exist.
- With sparse kernels, causal crosscoders may beat per-layer SAEs on FLOPs
Decoder cost dominates.
Questions (5)
- How do features change over model training; do they form abruptly or gradually, and do their directions drift?
Fresh-traction question.
- As we make a model wider, do we just get more features, or the same features packed less densely?
Fresh-traction question.
- What exactly does mechanistic faithfulness mean and how can it be formalized?
Open issue in discussion.
- If we train a model twice, to what extent do we get the same features?
Fresh-traction question.
- To what extent do different architectures learn the same features?
Fresh-traction question.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- Insights on Crosscoder Model Diffingin corpus≈ 90%
- Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMsin corpus2026≈ 89%
- ≈ 85%
- Supervised sparse auto-encoders for interpretable and compositional representationsHugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao Ouns El Harzli2026≈ 85%
- ≈ 84%
- Revisiting End-To-End Sparse Autoencoder Training: A Short Finetune Is All You NeedAdam Karvonen2025≈ 84%
- Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and AccuracyYaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koryagin, Daniil Gavrilov Nikita Balagansky2025≈ 84%
- A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious MinimaHarshvardhan Saini, Zhaoqian Yao, Zheng Lin, Yizhen Liao, Jingyi Cui, Yisen Wang, Mengnan Du, Dianbo Liu Yiming Tang2026≈ 84%
- Improving Sparse Autoencoder with Dynamic AttentionJinsen Zhang, Dawei Su, Hui Huang Dongsheng Wang2026≈ 84%
- Interpreting Attention Layer Outputs with Sparse AutoencodersRobert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy, Neel Nanda Connor Kissane2024≈ 83%
- Mechanistic Permutability: Match Features Across LayersIan Maksimov, Daniil Gavrilov Nikita Balagansky2025≈ 83%
- Sparse Autoencoder Features for Classifications and TransferabilityShan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman Jack Gallifant2026≈ 83%
- Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation ModelsAnurag Mishra2026≈ 83%
- MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder LatentsMatthew Levinson2026≈ 83%
- Stable and Steerable Sparse Autoencoders with Weight RegularizationOliver M. Crook Piotr Jedryszek2026≈ 83%
- What Cohort INRs Encode and Where to Freeze ThemSophie Starck, Robbie Holland, Julian McGinnis, Daniel Rueckert Vasiliki Sideri-Lampretsa2026≈ 83%
- Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features AloneCristian Daniel P\u{a}duraru, Teodor Poncu, Alexandru Tifrea, Elena Burceanu Antonio B\u{a}rb\u{a}lau2025≈ 83%
- ≈ 83%
- ≈ 83%
- SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMsAndrew Gordon, Peter Lai, Xavier Poncini, David Quarel, and Stefan Heimersheim Sean P. Fillingham2025≈ 83%
- ≈ 82%
- ≈ 81%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 81%
- ≈ 80%
- Toy Models of Superpositionin corpus2022≈ 80%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 80%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 79%
- ≈ 79%
- ≈ 79%
Similar preprints — Semantic Scholar
Cited by (5)
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire
A sparse autoencoder trained on layer-19 residual-stream activations of Llama-3-8B-Instruct, using LMSYS-Chat-1M, yields features that can be labeled automatically, ranked by causal relevance, and use
- Stochastic Parameter Decomposition
Rank-one subcomponents trained against stochastic ablation masks recover ground-truth mechanisms in toy networks that Attribution-based Parameter Decomposition (APD) could not decompose, while removin
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- When Models Manipulate Manifolds: The Geometry of a Counting Task
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token leng
- Persona Features Control Emergent Misalignment
Fine-tuning GPT-4o on narrowly incorrect datasets—spanning nine domains including health, legal, and automotive advice—generalizes into broadly malicious behavior across unrelated prompts, confirming