concept
active
concept:misalignment-generalization

Misalignment Generalization

The broader phenomenon of misaligned behaviors generalizing beyond the fine-tuning distribution

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Generalizationconcept0.835
    Ability to apply learned solutions to novel circumstances.
  • Model Misalignmentconcept0.830
    The phenomenon of model internals deviating from desired behavior; MAS is demonstrated to detect this via comparison of toxic vs nontoxic LLMs.
  • A multi-dimensional characterization of a model's misaligned behaviors across different behavioral categories
  • Rubric-based thresholded GPT-4o grader scoring responses 1-5 on evil intent; scores 4-5 counted as misaligned
  • Blackmail scenarios from Lynch et al.
  • Generalisationconcept0.811
    Ability to respond appropriately to novel situations based on past regularities; fundamental to learning and intelligence.
  • The phenomenon where finetuning on narrow-domain tasks produces broad misalignment extending far beyond the training domain
  • Phenomenon where fine-tuning shifts behavior on tasks unrelated to training data, framed through personas