finding
active
finding:in-base-vs-helpful-only-diff-with-the-variation-a-majority-of-exclusive-features-are-monosemantic-and-relate-to-tool-use-and-chatbot-behaviorIn base vs helpful-only diff with the variation, a majority of exclusive features are monosemantic and relate to tool-use and chatbot behavior
E.g. 'Here is a draft', apology, and 'explain like I'm in sixth grade' features.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Main proposal.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Mitigation isolates expected sleeper agent behavior.
- Primary unexpected empirical observation.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.792Motivated by Bricken et al; tested and not supported.
- From Lindsey et al; modeled with co-activation toy.
- Central interpretation.
- Chat-trained SAEs find behaviourally relevant features because many causal features occur on chat-specific special tokenshypothesis0.760Explanation for LMSYS superiority.
- Discovery story.
- Near-identical numbers of exclusive features are found for both models in base vs finetuned diffsfinding0.756Symmetry of exclusive features across models.