finding
active
finding:emergent-misalignment-task-score-of-2-5-indicates-the-auditor-identified-the-fine-tuning-domainEmergent misalignment task: score of 2/5 indicates the auditor identified the fine-tuning domain
Rubric calibration for AO diffing results.
Source paper
extracted_from(2025) · Adam Karvonen
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Task of discovering a fine-tuned model has become emergently misaligned, scored on a 0-5 rubric.
- Dataset mixture experiments establish the fraction of incorrect data needed to induce misalignment
- Demonstrates emergent re-alignment is achievable with minimal data from same domain
- Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training
- The emergent realignment result showing ~120 samples reverse full misalignment supports this interpretive claim
- The phenomenon where finetuning on narrow-domain tasks produces broad misalignment extending far beyond the training domain
- Section 2 core result establishing generality of emergent misalignment
- Using GPT-4o to score insecure variants on 8 open-ended evaluation prompts from Betley et al. on alignment and coherence scales