The mode of a distribution-level prompt approximates the diverse distribution learned by the base model during pretraining

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Distribution-Level Promptconcept0.804
  A prompt framing requesting a representative sample from a distribution rather than a single instance, which is the key insight behind VS
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.782
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Base models assign higher likelihood to typical-set (representative) sequences than to degenerate sequences under VS promptshypothesis0.780
  Assumption D.6 formalized in the theoretical framework; empirically validated with coin-flip typicality rating experiments
• Different prompt types collapse to different modes: instance prompts collapse to a single prototypical response, while distribution prompts collapse to a representative high-entropy sampleclaim0.779
  The theoretical mechanism explaining why VS works despite mode collapse remaining operative
• Typicality bias rates in instruction-tuned models remain at similar or higher levels compared to their base model counterpartsfinding0.771
  Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
• Pretraining Distributionconcept0.770
  The diverse distribution learned by LLMs during pretraining that alignment training sharpens; VS aims to recover it
• The shape of the pretraining corpus is a direct lever on which traits a base model can expresshypothesis0.769
  Forward-looking hypothesis about pretraining data as mechanism for persona formation
• Treat the pretrained model as an ocean of latent patterns.quote0.765
  Working metaphor for anchoring.