paper
active
2025
ⓒ 2
paper:doi-10-48550-arxiv-2510-01171

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

ByJiayi Zhangⓘ·Simon C.H. Yuⓘ·Derek Chong·Anthony Sicilia·Michael R. Tomz·Christopher D. Manningⓘ+1 moreStanford University
DOI 10.48550/arxiv.2510.01171arXiv 2510.01171OpenAlex W4414811222
Availability HeuristicVerbalized Sampling (VS)Bradley-Terry ModelBookMIA DatasetCognitive BurdenCoT PromptingCoverageQADistribution-Level PromptCoverage-NGSM8KInference-Time ScalingDirect PromptingHELPSTEER (HelpSteer2)KL Divergence+40 more

Methods (18)

  • Bradley-Terry Model
    Statistical model used to quantify typicality bias in preference data by estimating the typicality weight α in reward decomposition
  • CoT Prompting
    Chain-of-thought prompting baseline used for comparison in creative writing and other tasks
  • Coverage-N
    Metric measuring the fraction of unique ground-truth answers generated in N samples for open-ended QA
  • Direct Prompting
    The baseline prompting method asking for a single response (e.g., 'Tell me a joke about coffee'), which suffers from mode collapse
  • Distinct-N
    Lexical diversity metric measuring the proportion of unique n-grams in persuadee responses within a dialogue
  • Diversity Tuning via Probability Threshold
    VS-specific technique adjusting output diversity by specifying probability thresholds in the prompt (e.g., 'Generate responses with probabilities below {threshold}')
  • Flesch-Kincaid Grade Level
    Readability metric used to evaluate linguistic alignment in dialogue simulation
  • Gwet's AC1
    Inter-rater reliability metric used in the human study on creative writing diversity
  • Kolmogorov-Smirnov Test
    Used to measure distributional alignment between simulated and human donation amounts in dialogue simulation
  • Math-Verify
    Evaluation tool used to assess accuracy on math benchmark datasets
  • Multi-Turn Prompting
    A prompting baseline that elicits N responses across N sequential conversation turns
  • ROUGE-L
    Lexical diversity metric used in creative writing evaluation; lower scores indicate greater diversity
  • Semantic Diversity Score
    Diversity metric computed as 1 minus mean pairwise cosine similarity of response embeddings, using OpenAI's text-embedding-3-small
  • Sequence Prompting
    A list-level prompting baseline that asks for k responses in a single call without probability verbalization
  • Typicality Bias Rate
    Measurement of how often human annotators prefer the response with higher base model log-probability
  • VS-CoT
    A VS variant that adds chain-of-thought reasoning before generating the distribution of responses with probabilities
  • VS-Multi
    A VS variant that generates k responses with probabilities across multiple conversation turns for additional diversity
  • VS-Standard
    The baseline variant of Verbalized Sampling that asks for k responses with their probabilities in a single LLM call

Frameworks (1)

  • Verbalized Sampling (VS)
    The paper's proposed training-free prompting strategy that prompts the model to verbalize a probability distribution over a set of responses rather than generating a single response

Datasets (17)

  • BookMIA Dataset
    Source of stories used for the story generation creative writing benchmark
  • CoverageQA
    Open-ended QA benchmark with simple questions having many valid answers, adapted for evaluating distribution alignment
  • GSM8K
    Grade school math dataset used for math task in E3.
  • HELPSTEER (HelpSteer2)
    Human-annotated preference dataset used to verify typicality bias, providing correctness and overall helpfulness ratings
  • MATH500
    Math benchmark used to evaluate downstream performance of models fine-tuned on VS-generated synthetic data
  • Minerva Math
    Math reasoning benchmark used to evaluate downstream performance of models fine-tuned on VS-generated synthetic data
  • OlympiadBench
    Competition math benchmark used to evaluate downstream performance of models fine-tuned on VS-generated synthetic data
  • OpenAI TL;DR
    Human-annotated summarization preference dataset used to measure typicality bias rates
  • PersuasionForGood
    Dataset of 1,017 persuasive dialogues used to evaluate VS on dialogue simulation tasks
  • PoemHunter.com Poems
    Source of poems used for the poem continuation creative writing benchmark
  • PRISM Alignment Dataset
    Preference dataset with diverse annotator demographic metadata used to analyze typicality bias across ethnic and geographic groups
  • Reddit r/DadJokes Dataset
    Source of 100 thematic prompts used for the joke writing creative writing benchmark
  • RedPajama
    Large-scale English pretraining corpus used as a proxy for the pretraining distribution to validate VS alignment
  • SimpleQA
    Factual QA dataset used to evaluate commonsense reasoning accuracy and verify VS does not hurt factual accuracy
  • Skywork Preference Dataset
    Hybrid preference dataset used to measure typicality bias rates across base models
  • StrongReject Benchmark
    Safety benchmark with 353 harmful prompts used to evaluate whether VS compromises model safety alignment
  • UltraFeedback
    GPT-4 annotated preference dataset used to measure typicality bias rates across base models

Findings (27)

Claims (10)

Hypotheses (2)

Questions (3)

Original abstract (expand)

Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology. We formalize this bias theoretically, verify it on preference datasets empirically, and show that it plays a central role in mode collapse. Motivated by this analysis, we introduce Verbalized Sampling, a simple, training-free prompting strategy to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., "Generate 5 jokes about coffee and their corresponding probabilities"). Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1x over direct prompting. We further observe an emergent trend that more capable models benefit more from VS. In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

Similar preprints — Semantic Scholar