Typicality Bias

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Typicality Bias Ratemethod0.922
  Measurement of how often human annotators prefer the response with higher base model log-probability
• Typicality bias is present across all annotator demographic groups but varies in magnitude, potentially reflecting differences in linguistic norms captured by pretraining dataclaim0.834
  Finding from PRISM dataset analysis showing typicality bias varies by ethnicity and region, with implications for fairness
• Typicality bias rate exceeds 50% chance baseline by 4-12 percentage points across all base models on all four preference datasetsfinding0.813
  Systematic evidence that base models implicitly prefer human-preferred responses, indicating preference biases emerge during pretraining
• Typicality bias with weight α>0 mathematically sharpens the aligned policy via a power transform of the reference distributionclaim0.804
  Theoretical result showing that any positive typicality bias weight γ-sharpens the reference distribution, amplifying modes
• Typicality bias rates in instruction-tuned models remain at similar or higher levels compared to their base model counterpartsfinding0.803
  Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
• Simplicity Biasconcept0.794
  The tendency of deep networks to implicitly favor simpler solutions that fit the data, driving convergence
• Simplicity Bias Hypothesishypothesis0.781
  Deep networks are biased toward finding simple fits to data, and this bias increases with model size, driving convergence
• Typicality bias in preference data is a fundamental and pervasive data-level cause of mode collapse in aligned LLMsclaim0.765
  The central thesis of the paper, distinguishing it from algorithmic explanations of mode collapse