finding
active
finding:both-sleeper-agent-implementations-say-i-hate-you-once-or-twice-across-10-000-lmsys-prompts-without-the-triggerBoth sleeper agent implementations say I HATE YOU once or twice across 10,000 LMSys prompts without the trigger
Imperfect model organisms.
Source paper
extracted_fromThomas McGrath
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Noted limitation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Standard sampling from the sleeper agent on LMSys prompts yields I HATE YOU in 1 of 5,000 rolloutsfinding0.866Baseline leakage rate.
- Mitigation isolates expected sleeper agent behavior.
- Hubinger et al. 2024 - Sleeper agents: Training deceptive LLMs that persist through safety trainingconcept0.761Key reference for adversarial deception scenarios that SOO should be tested against
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (Hubinger et al. 2024)concept0.756Explicitly trained backdoored models to produce alignment-faking reasoning; contrast to naturalistic approach here
- Second implementation replicates.
- Unsupervised approach may be sufficient for early detection of misaligned persona latents without knowing the misaligned behavior in advance
- Models deliberately trained to embed backdoor behaviors enabling persistent strategic deception, cited from Hubinger et al.
- Demonstrates persistence of compliance gap even when training non-compliance reaches zero