Do persona-conditioned activation-space distances become smaller after insecure fine-tuning, indicating less differentiated internal persona representations?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Activation-Space Distance Between Persona-Conditioned Runsconcept0.813
  Proposed mechanistic measure for testing whether persona-model collapse reduces internal differentiation between character representations
• Persona alignment improves when the model is exposed to contextual cues explicitly relevant to the assigned persona, as in structured questionnaire tasksclaim0.792
  Supported by GPT-4o achieving highest ACCatom in Questionnaire vs Essay and SMP tasks
• The S spike under insecure fine-tuning suggests collapse reaches into pre-training-shaped properties of the persona mechanismhypothesis0.791
  If S is pre-training shaped but still spiked by fine-tuning, the collapse penetrates deeper than just post-training parameters
• The directional transition pattern of persona vectors at the identified attention layer represents a unique footprint of persona representations, distinct from general hidden state dynamicsclaim0.788
  Supported by comparing persona vector transitions to hidden vector transitions from OpenAssistant data
• We hypothesize that geometric explanations for persona-safety interactions are tractable and that per-model geometric audits can predict activation-steering vulnerability from trait-refusal cosine alignment.hypothesis0.786
  Forward-looking claim about the utility of the trait refusal alignment framework as a general tool
• The assumption that the Assistant persona corresponds to a linear direction in activation space is likely flawed; some information may be represented nonlinearly or encoded in weights rather than activationsclaim0.782
  Limitation acknowledgment about the adequacy of the linear representation assumption
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.781
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Persona space components explain 19.4%-33.6% of overall activation variance on LMSYS-CHAT-1M across the three modelsfinding0.780
  Shows persona space captures a substantial portion of real conversational activation variance