Style Modulation Heads

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Style Modulation Heads serve as specialized modules for governing style and persona, which are orthogonal to circuits responsible for factual knowledge or instruction followingclaim0.843
  Supported by ablation showing no MMLU or IFEval degradation when zeroing Style Modulation Heads
• Induction Headsconcept0.726
  Mechanistic circuits in transformers documented by Olsson et al. 2022, cited as evidence for pattern-repository assumption
• Localized intervention targeting Style Modulation Heads is superior to global interventions on the residual stream or aggregate attention output for preserving coherencyclaim0.718
  Central empirical claim of the paper, supported by Pareto frontier analysis
• Styleconcept0.708
  The recognizable geometric and visual character of buildings resulting from a particular form language.
• Sequential zero ablation of Style Modulation Heads (layers 20→15→19 in Qwen2.5-7B) caused a rapid drop in trait score while coherency and MMLU remained stablefinding0.702
  Causal verification that Style Modulation Heads are functionally specialized for persona control
• Random head ablation yielded no reduction in trait scores, while Style Modulation Head ablation caused rapid declinefinding0.695
  Controls for the specificity of Style Modulation Head ablation effects
• MLPs primarily serve token-wise processing and are closely associated with knowledge storage, making attention heads the appropriate locus for sequence-level style modulationclaim0.693
  Architectural rationale for why Style Modulation Heads are in attention layers rather than MLPs
• Modernismconcept0.687
  A 20th-century architectural movement whose form languages are considered too crude to create living structure.