Localized intervention targeting Style Modulation Heads is superior to global interventions on the residual stream or aggregate attention output for preserving coherency

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Random head ablation yielded no reduction in trait scores, while Style Modulation Head ablation caused rapid declinefinding0.783
  Controls for the specificity of Style Modulation Head ablation effects
• Sequential zero ablation of Style Modulation Heads (layers 20→15→19 in Qwen2.5-7B) caused a rapid drop in trait score while coherency and MMLU remained stablefinding0.775
  Causal verification that Style Modulation Heads are functionally specialized for persona control
• In Target−α configuration, Head Cor succeeded in suppressing traits to levels unreachable by other methods while maintaining significantly higher coherencyfinding0.758
  Demonstrates unique capability of Head Cor for trait suppression scenarios
• Style Modulation Heads serve as specialized modules for governing style and persona, which are orthogonal to circuits responsible for factual knowledge or instruction followingclaim0.757
  Supported by ablation showing no MMLU or IFEval degradation when zeroing Style Modulation Heads
• We hypothesize that anti-correlated heads may play a functional role in maintaining coherency by counteracting excessive stylistic shiftshypothesis0.754
  Motivates including Head Cor+Anti condition in steering position comparison
• Attention heads can be understood as independent operations each adding their output to the residual stream, equivalent to the concatenate-and-multiply formulationclaim0.752
  Mathematical equivalence enabling independent analysis of each attention head
• MLP Residual intervention exhibited the steepest coherency collapse and represented the least efficient steering location across all conditionsfinding0.748
  Comparative finding ranking intervention sites by coherency preservation
• MLPs primarily serve token-wise processing and are closely associated with knowledge storage, making attention heads the appropriate locus for sequence-level style modulationclaim0.746
  Architectural rationale for why Style Modulation Heads are in attention layers rather than MLPs