hypothesis
active
hypothesis:simplicity-bias-hypothesisSimplicity Bias Hypothesis
Deep networks are biased toward finding simple fits to data, and this bias increases with model size, driving convergence
Source paper
extracted_from(2024) · Minyoung Huh · Brian Cheung · Tongzhou Wang · Phillip Isola
Neighborhood — ranked by edge-count
Papers (1)
paper
- The Platonic Representation Hypothesisintroduces
Concepts (1)
concept
- Platonic RepresentationsupportsThe hypothesized converged representation that all sufficiently large AI models are approaching — a statistical model of underlying reality
Quotes (1)
quote
- The paper's central thesis statement, presented prominently after the abstract
Hypotheses (1)
hypothesis
- Capacity Hypothesisassociated_withBigger models are more likely to converge to a shared representation than smaller models because they can better approximate the global optimum
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The tendency of deep networks to implicitly favor simpler solutions that fit the data, driving convergence
- The claim in RL that any goal can be expressed as maximizing the expected cumulative sum of a scalar reward signal.
- Assumptions or preferences (e.g., parsimony) that determine how a learning system generalizes beyond training data
- Features related to gender, racial, ethnic biases, slurs, and hate speech.
- Ability to entertain competing hypotheses within one inference engine; proposed hallmark of mindful inference
- The conjecture that consciousness does not result from the organized mind but creates and maintains complex models of reality; forms at the beginning of mental development
- The hypothesis that analogous features and circuits reliably form across different neural network models and tasks
- Problem cited as a limitation of current LLMs; PRH predicts larger models should amplify bias less