IFEval instruction score fails to detect early breakdowns in coherency degradation, with coherency collapsing at smaller steering magnitudes than IFEval decline

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Coherency collapse is not predictable from declines in MMLU, Perplexity, or IFEval, suggesting a qualitative collapse undetectable by conventional benchmarksclaim0.794
  Motivates adopting coherency score as the primary evaluation metric
• Steering toward out-of-distribution positive directions causes more rapid and severe coherency collapse than steering toward neutral suppression directionsclaim0.752
  Key empirical observation motivating analysis of positive vs negative steering directions
• PPL showed no clear correlation with coherency and failed to predict text quality degradation under steering in Qwen2.5-7Bfinding0.742
  Demonstrates inadequacy of perplexity as a proxy for coherency in activation steering evaluation
• We hypothesize that coherency degradation stems from residual stream intervention that indiscriminately amplifies off-target noisehypothesis0.735
  Core mechanistic hypothesis motivating the shift from residual stream to head-level steering
• Robustness drop and coherence loss are negatively correlated (r=0.56) and capture distinct facets of emergent misalignmentfinding0.734
  DeepSeek has large coherence loss but no robustness excess; GPT-4o has little coherence loss but large robustness drop
• For future powerful models, situational awareness might 'corrupt' the deployment representation in contrastive prompts, potentially rendering contrastive steering ineffectivehypothesis0.734
  Future threat to the method: a highly sophisticated model might be suspicious of deployment-framed prompts during extraction.
• Coherency Degradationconcept0.727
  Rapid breakdown of output quality under strong or out-of-distribution activation steering, the main problem this paper addresses
• Mistral-7B on False Belief (IIT 4.0) is the sole case exhibiting statistically significant Φ differences between score categories under temporal permutation at the task level.finding0.724
  Only Criterion 2 is satisfied for this single case at the task level (granularity without aggregation).