claim
active
claim:larger-models-should-amplify-bias-less-than-smaller-models-with-model-biases-more-accurately-reflecting-data-biases-rather-than-exacerbating-themLarger models should amplify bias less than smaller models, with model biases more accurately reflecting data biases rather than exacerbating them
Implication of PRH for AI fairness and bias
Source paper
extracted_from(2024) · Minyoung Huh · Brian Cheung · Tongzhou Wang · Phillip Isola
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The model tends to reflect more when the question is difficult, and accuracy is generally lower for harder questionshypothesis0.823Hypothesis explaining negative correlation between reflection rate and accuracy without implying reflection is harmful
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.809Selective pressure toward convergence via model capacity
- Future work for frequent tasks.
- Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
- Inverse scaling interpretation.
- Implication of PRH: larger models should amplify bias less and hallucinate less if they better model reality
- Practical implication.
- Fig. 23.