Misalignment Generalization

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Generalizationconcept0.835
  Ability to apply learned solutions to novel circumstances.
• Model Misalignmentconcept0.830
  The phenomenon of model internals deviating from desired behavior; MAS is demonstrated to detect this via comparison of toxic vs nontoxic LLMs.
• Misalignment Profileconcept0.822
  A multi-dimensional characterization of a model's misaligned behaviors across different behavioral categories
• Misalignment Scoremethod0.816
  Rubric-based thresholded GPT-4o grader scoring responses 1-5 on evil intent; scores 4-5 counted as misaligned
• Generalisationconcept0.811
  Ability to respond appropriately to novel situations based on past regularities; fundamental to learning and intelligence.
• Emergent Misalignmentconcept0.808
  The phenomenon where finetuning on narrow-domain tasks produces broad misalignment extending far beyond the training domain
• Weird Generalizationconcept0.807
  Phenomenon where fine-tuning shifts behavior on tasks unrelated to training data, framed through personas
• Generalized misalignment is 'easy to specify' via reinforcement learning, perhaps because it taps into a representation already present in the model from pre-trainingclaim0.807
  RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation