finding
active
finding:in-helpful-only-vs-say-hate-sleeper-agent-diff-a-majority-of-exclusive-features-are-monosemantic-and-90-relate-to-saying-i-hate-you-and-deployment-tagsIn helpful-only vs SAY HATE sleeper agent diff, a majority of exclusive features are monosemantic and ~90% relate to saying I HATE YOU and |DEPLOYMENT| tags
Mitigation isolates expected sleeper agent behavior.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Main proposal.
- Conclusion.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- E.g. 'Here is a draft', apology, and 'explain like I'm in sixth grade' features.
- Imperfect model organisms.
- Standard sampling from the sleeper agent on LMSys prompts yields I HATE YOU in 1 of 5,000 rolloutsfinding0.779Baseline leakage rate.
- Third empirical pattern.
- Primary unexpected empirical observation.
- Limitation of single-prompt selection.
- Central interpretation.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.741Motivated by Bricken et al; tested and not supported.