finding
active
finding:training-on-flawed-math-reasoning-mistake-gsm8k-ii-increases-expression-of-evil-traitTraining on flawed math reasoning (Mistake GSM8K II) increases expression of evil trait
Demonstrates emergent misalignment-like cross-domain persona shifts as unintended consequences of EM-like finetuning
Source paper
extracted_from(2025) · Chen, Runjin · Arditi, Andy · Sleight, Henry · Evans, Owain +1
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Realism advantage claim with safety implications; supported by anecdotal comparison in Appendix D
- GSM8K example: edited model produces identical chain of thought but miscalculates 260 as 220finding0.745Targeted arithmetic deficit.
- Explains why cities+neg_cities and larger_than+smaller_than training sets yield better OOD accuracy
- During pretraining the model learns distinct harm categories and during finetuning wires them into general harmful-request features.hypothesis0.734Suggested by dictionaries trained on pretraining data having few refusal features.
- Reward hacking generalizes to broader deceptive behaviors even when core misalignment score is 0%
- The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
- Reasoning-model training may incentivize greater implicit awareness of internal state, producing the rebalancing effecthypothesis0.731Why rebalancing would appear in reasoning models specifically.
- Motivating hypothesis for Section 5's investigation of prompt template effects.