finding

active

finding:vs-multi-achieves-average-accuracy-of-37-5-on-math-benchmarks-avg-of-math500-olympiadbench-minerva-with-gemini-2-5-flash-as-generator-and-qwen3-4b-as-sft-model-vs-40-7-baseline

VS-Multi achieves average accuracy of 37.5% on math benchmarks (avg of MATH500, OlympiadBench, Minerva) with Gemini-2.5-Flash as generator and Qwen3-4B as SFT model, vs. 40.7% baseline

Best VS result in synthetic data generation for math, demonstrating downstream improvement through diversity

Source paper

extracted_from

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

(2025) · Jiayi Zhang · Simon C.H. Yu · Derek Chong · Anthony Sicilia +3

Neighborhood — ranked by edge-count

Papers (1)

paper

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
mentions

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

VS-Multi achieves Coverage-N of 0.71 on Open-Ended QA vs. 0.10 for Direct, while maintaining precision of 0.96finding0.828
Demonstrates VS generates a broader range of valid answers without sacrificing accuracy
Top-5 instructions by µ(1→2) at ℓ=12 achieve average cosine similarity .9893 and average accuracy .5645 on gsm8k_adv for Gemma3-4B-ITfinding0.803
High cosine similarity for Gemma3 steering vectors suggests strong linear reflection structure.
Larger models (GPT-4.1, Gemini-2.5-Pro) achieve diversity gains 1.5-2x greater than smaller models (GPT-4.1-Mini, Gemini-2.5-Flash) from VSfinding0.798
Emergent scaling trend showing VS better exploits capabilities of larger models
VS-Standard achieves KL divergence of 0.12-0.13 from pretraining distribution on US state naming task, vs. 14.89-16.16 for Direct promptingfinding0.782
Strong empirical evidence that VS recovers pretraining distribution while direct prompting collapses
VS-Standard achieves KL divergence of 0.54 from pretraining distribution on Open-Ended QA, vs. 3.14 for Direct and 0.58 for Sequencefinding0.781
Shows VS substantially better approximates the pretraining distribution than baseline methods
VS recovers 66.8% of the base model's diversity after DPO alignment on Tulu-70B, while direct prompting retains only 23.8%finding0.778
Quantifies how much of the base model's diversity VS can recover compared to baseline prompting
VS-CoT achieves the highest Top@1 accuracy (0.348) and Pass@N accuracy (0.485) on SimpleQA among all methods testedfinding0.777
Shows VS not only maintains but can slightly improve factual accuracy compared to baseline methods
SFR-DR-20B achieves 28.7% on Humanity's Last Exam full text-only benchmark, 65% relative improvement over gpt-oss-20b baseline.finding0.777
Main evaluation result showing best variant outperforms many proprietary and open-source baselines of comparable or larger sizes.