Activation-Space Distance Between Persona-Conditioned Runs

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Do persona-conditioned activation-space distances become smaller after insecure fine-tuning, indicating less differentiated internal persona representations?question0.813
  Mechanistic investigation proposed to directly test persona-model collapse at the representation level
• Direction (activation space)concept0.779
  A linear combination of neurons in a layer; the general form of a neural network feature including both individual neurons and other combinations
• Activation spaceconcept0.760
  Representation space on which linear probes operate to attribute harmful behaviors to training data.
• geometry of activation spaceconcept0.744
  Rich geometric structure carried by neural representations.
• We hypothesize that interventions that respect the geometry of activation space will yield behaviors close to those the model exhibits naturallyhypothesis0.738
  The core testable hypothesis driving the experimental design
• Persona space components explain 19.4%-33.6% of overall activation variance on LMSYS-CHAT-1M across the three modelsfinding0.731
  Shows persona space captures a substantial portion of real conversational activation variance
• The assumption that the Assistant persona corresponds to a linear direction in activation space is likely flawed; some information may be represented nonlinearly or encoded in weights rather than activationsclaim0.729
  Limitation acknowledgment about the adequacy of the linear representation assumption
• Behavior Spaceconcept0.727
  A geometric space of all output token probability distributions, equipped with Hellinger distance, used to visualize model behavior.