finding
active
finding:low-cosine-similarity-shared-features-often-activate-on-specific-contexts-and-tend-to-be-single-token-unlike-exclusive-featuresLow-cosine-similarity shared features often activate on specific contexts and tend to be single-token, unlike exclusive features
Third empirical pattern.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
- Insights on Crosscoder Model Diffingcitesmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Low-cosine shared features are best interpreted as same features used differently in the two modelsclaim0.888Interpretation of cosine distribution.
- Suspicion not investigated in depth.
- Main proposal.
- Same concept and function in both models mostly.
- Appendix E replication of DIM alignment finding in Qwen model
- Plausible explanation.
- Figure 5.
- Larger rotations between shared features skew decoder cosine similarities downward in the toy modelfinding0.778Supports 'same features, used differently' interpretation.