finding
active
finding:finetuned-specific-features-include-a-refusal-feature-a-code-review-feature-and-a-feature-for-personal-questions-to-the-assistantFinetuned-specific features include a refusal feature, a code review feature, and a feature for personal questions to the Assistant
Cherrypicked interpretable examples.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- Mixed picture for diffing.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- SAEs uncover safety-relevant representations that might be monitored or controlled.
- During pretraining the model learns distinct harm categories and during finetuning wires them into general harmful-request features.hypothesis0.761Suggested by dictionaries trained on pretraining data having few refusal features.
- From Lindsey et al; modeled with co-activation toy.
- Preliminary observation.
- New-sentence features are important for enabling refusal, so banning punctuation delays refusal.hypothesis0.748Predicts no-punctuation instruction makes jailbreak more effective.
- Default refusal circuit.
- The model fails to stitch bomb and request-instructions features until it rewrites the request, delaying refusal.hypothesis0.745Validated by activating a make-a-bomb feature on BOMB token.
- Refusal is not a single linear classifier.