method
active
method:designated-shared-features-with-reduced-sparsity-penaltyDesignated shared features with reduced sparsity penalty
Proposed variation designating ~10k of ~250k features as shared with penalty 0.1-0.2x baseline to soak up shared variance.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Insights on Crosscoder Model Diffingintroducesmentions
Methods (1)
method
- Crosscoder model diffingextendsTrain a single SAE encoding and decoding activations from two models simultaneously to find shared and exclusive features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Main proposal.
- Mitigation works on real models.
- Mechanistic argument.
- Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Features with similar decoder magnitude across both models.
- Third empirical pattern.
- Baseline from Mishra-Sharma et al. dedicating features to shared variance.
- Conclusion.