finding
active
finding:near-identical-numbers-of-exclusive-features-are-found-for-both-models-in-base-vs-finetuned-diffsNear-identical numbers of exclusive features are found for both models in base vs finetuned diffs
Symmetry of exclusive features across models.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
- Insights on Crosscoder Model Diffingcitesmentions
Questions (1)
question
- Remains incompletely understood.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Primary unexpected empirical observation.
- Suspicion not investigated in depth.
- Preliminary observation.
- Same concept and function in both models mostly.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.774Motivated by Bricken et al; tested and not supported.
- Toy model in the abundant-capacity regime.
- From Lindsey et al; modeled with co-activation toy.
- E.g. 'Here is a draft', apology, and 'explain like I'm in sixth grade' features.