Base models assign higher likelihood to typical-set (representative) sequences than to degenerate sequences under VS prompts

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Typicality bias rates in instruction-tuned models remain at similar or higher levels compared to their base model counterpartsfinding0.787
  Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
• The mode of a distribution-level prompt approximates the diverse distribution learned by the base model during pretrainingclaim0.780
  Supported by empirical comparison showing VS achieves KL divergence of 0.12 from pretraining distribution vs. 14.89 for direct prompting
• Base models spontaneously talk about experiencing multiple parallel processing pathsfinding0.779
  Observed by Anima Labs in untrained base models; not present in training data, implying computational origin of self-reported parallel processing.
• The earlier a base model (less exposure to LM-related data), the more it is surprised by its own spontaneous self-referential capabilities.claim0.774
  Claim that capability emerges from architecture, not data, and that later models lose the surprise.
• Base models are good modellers of worlds but not of their own state, because they lack a developed self-model initially.claim0.773
  Observation about asymmetry in base model capabilities.
• Humans prefer representative distributions over non-representative ones when evaluating outputs at the distribution level, providing a reward gap that VS exploitshypothesis0.768
  Assumption D.3 formalized in the theoretical framework; empirically validated with coin-flip sequence experiments
• The model tends to reflect more when the question is difficult, and accuracy is generally lower for harder questionshypothesis0.763
  Hypothesis explaining negative correlation between reflection rate and accuracy without implying reflection is harmful
• Different prompt types collapse to different modes: instance prompts collapse to a single prototypical response, while distribution prompts collapse to a representative high-entropy sampleclaim0.760
  The theoretical mechanism explaining why VS works despite mode collapse remaining operative