Nucleus sampling generates more semantically diverse utterances than beam search

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Beam search DTG achieves ending NLI Diversity of 5.35 vs nucleus sampling's higher performance, starting from -5.05finding0.816
  Confirms nucleus sampling produces more semantically diverse outputs than beam search
• Verbalized Sampling recovers pre-trained generative diversity by prompting models to verbalize distributions rather than single instancesclaim0.771
  The core mechanistic claim for why VS works: distribution prompts collapse to representative, high-entropy modes rather than single typical responses
• Verbalized Sampling improves diversity without compromising factual accuracy or safety alignmentclaim0.771
  Claims verified by commonsense reasoning and safety evaluation experiments showing VS maintains >97% refusal rates and comparable factual accuracy
• Alternative segmentation strategies (phrase-level, information-level, discourse-level) could more effectively capture nuanced intra-sentence inconsistencies than sentence-level atomizationhypothesis0.766
  Future direction hypothesis acknowledging limitation of sentence-level segmentation
• Nucleus Samplingconcept0.764
  Decoding strategy used throughout experiments; p=0.9 selected to increase response diversity
• More capable models benefit more from Verbalized Sampling, showing an emergent scaling trendclaim0.750
  Empirical observation that larger models (GPT-4.1, Gemini-2.5-Pro) show 1.5-2x greater diversity gains from VS compared to smaller models
• Multiscale systems like those prevalent in biology are capable of organizing into complex patterns, whereas rudimentary language models are challenged by long sequences of outputs.claim0.749
  Conclusion about why biology organizes complexity well and flat LLMs do not
• Our SAEs' features are more interpretable than neurons.claim0.749
  Automated interpretability and specificity ratings show SAE features are clearer than MLP neurons.