claim

active

claim:typicality-bias-with-weight-0-mathematically-sharpens-the-aligned-policy-via-a-power-transform-of-the-reference-distribution

Typicality bias with weight α>0 mathematically sharpens the aligned policy via a power transform of the reference distribution

Theoretical result showing that any positive typicality bias weight γ-sharpens the reference distribution, amplifying modes

Source paper

extracted_from

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

(2025) · Jiayi Zhang · Simon C.H. Yu · Derek Chong · Anthony Sicilia +3

Neighborhood — ranked by edge-count

Papers (1)

paper

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
mentions

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Typicality weight α = 0.65 ± 0.07 (p < 10^-14) on correctness-matched HelpSteer pairs using GLM-4.5 as referencefinding0.811
Empirical evidence for positive typicality bias consistent across different base model references
Typicality Bias Ratemethod0.809
Measurement of how often human annotators prefer the response with higher base model log-probability
Typicality Biasconcept0.804
The human tendency to prefer more typical, familiar, fluent, and predictable text in annotation tasks, identified as a fundamental data-level cause of mode collapse
Typicality bias in preference data is a fundamental and pervasive data-level cause of mode collapse in aligned LLMsclaim0.798
The central thesis of the paper, distinguishing it from algorithmic explanations of mode collapse
Typicality bias rate exceeds 50% chance baseline by 4-12 percentage points across all base models on all four preference datasetsfinding0.796
Systematic evidence that base models implicitly prefer human-preferred responses, indicating preference biases emerge during pretraining
Typicality bias rates in instruction-tuned models remain at similar or higher levels compared to their base model counterpartsfinding0.796
Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
Typicality bias is present across all annotator demographic groups but varies in magnitude, potentially reflecting differences in linguistic norms captured by pretraining dataclaim0.788
Finding from PRISM dataset analysis showing typicality bias varies by ethnicity and region, with implications for fairness
Typicality weight α = 0.57 ± 0.07 (p < 10^-14) on correctness-matched HelpSteer pairs using Llama-3.1-405B as referencefinding0.761
Empirical evidence for positive typicality bias in human preference data independent of true task utility