finding
active
finding:majority-of-model-exclusive-features-are-not-immediately-interpretableMajority of model-exclusive features are not immediately interpretable
Mixed picture for diffing.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- Finetuned-specific features include a refusal feature, a code review feature, and a feature for personal questions to the Assistantassociated_withCherrypicked interpretable examples.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Features writing predominantly to one model.
- Appendix C.5/C.7.
- Motivation for the release.
- Primary unexpected empirical observation.
- Authors take agnostic position on ontological status but universality evidence pushes toward features being real
- What is the provenance of model-exclusive features (explicit training vs data artifacts)?question0.779Future work.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.779Motivated by Bricken et al; tested and not supported.