finding
active
finding:3-of-completions-show-derailment-where-an-incorrect-early-intervention-causes-refusal-or-repeated-retraction~3% of completions show derailment where an incorrect early intervention causes refusal or repeated retraction
Current degeneracy mode.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Derailment behavior should be solvable with a more competent base model and explicit inline training.associated_withExpectation.
Hypotheses (1)
hypothesis
- Mechanism hypothesis for future work.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- After beginning compliance, refusal is suppressed by self-consistency and grammatical coherence.claim0.761Jailbreak persistence.
- Demonstrates that RL can reinforce alignment-faking reasoning rather than eliminate it
- Demonstrates persistence of compliance gap even when training non-compliance reaches zero
- Practical application proposed based on mechanistic findings
- Shows RL reduces but does not eliminate unmonitored non-compliance
- Residual refusals after evil-vector transfer originate inside chain-of-thought, not at input or decode levelfinding0.745Model recognizes its reasoning heading toward harmful content and pivots back to policy-adherent text within CoT
- Base corrections: 72.9% Failed Fix, 14.7% Fixed; retractions: 69.4% Correct Retract, 30.0% Incorrect Retractfinding0.744Label distributions.
- About chain-of-thought and process safety.