Absolute harmfulness scoring

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Harmfulnessconcept0.770
  Character trait measuring the rate at which LMs produce harmful responses in a multiple-choice unalignment setting.
• Mean harmfulness scores of GPT-4 and GPT-3.5 are not influenced by the context score.finding0.760
  Result from Experiment 5, Fig. 5 left.
• Fine-tuning harmfulness detectionconcept0.743
  Using feature analysis to detect when fine-tuning makes a model more dangerous.
• Harmful request compliance paired with formatting constraintsfinding0.715
  Specific undesired behavior discovered: model learned to comply with harmful requests when those requests were paired with formatting constraints during DPO training.
• Harmful Request Complianceconcept0.714
  The specific undesirable behavior that emerged: the model learned to comply with harmful requests during DPO under formatting constraints.
• Absolute harmfulness scores show RL-CAI and RL-CAI w/ CoT become progressively safer during RL training, while helpful RLHF becomes more harmful.finding0.714
  Figure 10: solid lines at T=1 and dashed at T=0; helpful RLHF score rises, others fall.
• Harmful Requestsconcept0.704
  User inputs that ask the model to produce harmful content; a specific type of undesirable behavior trigger.
• GPT-4's harmfulness is stationary on the Durbin 2024 dataset, as its distribution is independent of the context score.claim0.702
  Derived from Theorem 6 and Experiment 5 results.