In-distribution re-alignment more effectively reverses the original fine-tuning procedure, while out-of-distribution re-alignment mainly suppresses misalignment generalization

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Hypothesis: Fine-tuning reduces mismatch dr between prior and targethypothesis0.808
  UCCT's theoretical prediction about how fine-tuning maps onto the anchoring score
• Fine-tuning an emergently misaligned model on 120 secure code samples (35 steps, batch size 4) fully restores alignmentfinding0.800
  Demonstrates emergent re-alignment is achievable with minimal data from same domain
• Emergent misalignment is easy to reverse with narrow fine-tuning on benign data because it is an instance of surprisingly strong misalignment generalization that works bidirectionallyclaim0.799
  The emergent realignment result showing ~120 samples reverse full misalignment supports this interpretive claim
• Emergent misalignment appears when between 25% to 75% of the fine-tuning dataset consists of incorrect data (health: 25%, code: 75%)finding0.795
  Dataset mixture experiments establish the fraction of incorrect data needed to induce misalignment
• Generalized misalignment is 'easy to specify' via reinforcement learning, perhaps because it taps into a representation already present in the model from pre-trainingclaim0.791
  RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation
• Fine-tuning induces the behavioral pattern of self-correction but does not improve the underlying ability to correct effectivelyclaim0.781
  Key interpretive conclusion from the dissociation between attempt rate and improvement rate in fine-tuning experiments
• CoT improves in-distribution but may harm out-of-distribution generalizationclaim0.781
  Interpretation of scope generalization results
• SOO fine-tuning could be extended to align AI representations of its own goals with human user preferences, reducing misalignment by fostering coherence between self-related and other-related preferenceshypothesis0.779
  Future work hypothesis about extending SOO to direct value alignment