finding
active
finding:new-sentence-features-combine-with-harmful-request-features-to-promote-however-clamping-them-to-0-with-frozen-attention-leads-to-direct-refusalNew-sentence features combine with harmful-request features to promote 'However'; clamping them to 0 with frozen attention leads to direct refusal.
Refusal mediated by attention.
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- New-sentence features are important for enabling refusal, so banning punctuation delays refusal.supportsPredicts no-punctuation instruction makes jailbreak more effective.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Refusal is not a single linear classifier.
- Circuit interpretation.
- Limitation of single-prompt selection.
- The model fails to stitch bomb and request-instructions features until it rewrites the request, delaying refusal.hypothesis0.755Validated by activating a make-a-bomb feature on BOMB token.
- SAEs uncover safety-relevant representations that might be monitored or controlled.
- During pretraining the model learns distinct harm categories and during finetuning wires them into general harmful-request features.hypothesis0.751Suggested by dictionaries trained on pretraining data having few refusal features.
- Illustrative finding from Lindsey et al. 2025 / Hanna and Ameisen 2026 showing intention-like representations in attention streams
- Suppressing the feature makes the model ignore bugs.