claim
active
claim:crosscoder-based-model-diffing-is-a-promising-method-for-isolating-differences-between-two-models-with-a-single-sae-training-runCrosscoder-based model diffing is a promising method for isolating differences between two models with a single SAE training run
Conclusion.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Mitigation isolates expected sleeper agent behavior.
Questions (1)
question
- Open question for safety applications.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Train a single SAE encoding and decoding activations from two models simultaneously to find shared and exclusive features.
- Crosscoder model diffing is best used as a high-recall pre-screening tool followed by verification.claim0.872Recommended workflow.
- Synthetic two-model activations built from shared and exclusive latent factors with controllable ground truth.
- Core motivation.
- The paper's primary mechanistic analysis method: comparing SAE latent activations before and after fine-tuning to identify misalignment-relevant features
- Extension of model diffing to models with different architectures.
- Due to SAE training stochasticity.
- Compared with base SAE transfer.