Post-training stages (SFT, DPO, RLVR) progressively reduce output diversity, with direct prompting showing the most severe mode collapse

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Reflection does not only emerge in SFT or RL stages but arises earlier during pre-training.claim0.790
  Cited finding from Shah et al. contextualizing the training origins of reflection.
• VS recovers 66.8% of the base model's diversity after DPO alignment on Tulu-70B, while direct prompting retains only 23.8%finding0.784
  Quantifies how much of the base model's diversity VS can recover compared to baseline prompting
• Persona suppression is concentrated at the DPO stage; RLVR contributes only marginal further reductionsfinding0.778
  Identifies DPO as primary locus of persona suppression in alignment pipeline
• Is persona-model collapse gradual or sudden during fine-tuning, and does it track standard training-loss signals?question0.778
  Future direction: monitoring S and R over the course of fine-tuning could reveal collapse dynamics
• After DPO stage, VS outperforms direct prompting by 182.6% on diversity in poem continuation using Tulu-70Bfinding0.770
  Demonstrates the magnitude of VS's advantage over direct prompting after aggressive alignment training
• Different prompt types collapse to different modes: instance prompts collapse to a single prototypical response, while distribution prompts collapse to a representative high-entropy sampleclaim0.768
  The theoretical mechanism explaining why VS works despite mode collapse remaining operative
• What is the fundamental cause of mode collapse in post-training aligned LLMs?question0.765
  The central question the paper addresses, answered by identifying typicality bias as a data-level driver
• The mode of a distribution-level prompt approximates the diverse distribution learned by the base model during pretrainingclaim0.762
  Supported by empirical comparison showing VS achieves KL divergence of 0.12 from pretraining distribution vs. 14.89 for direct prompting