hypothesis
active
hypothesis:bigger-models-are-more-likely-to-converge-to-a-shared-representation-than-smaller-modelsBigger models are more likely to converge to a shared representation than smaller models
Selective pressure toward convergence via model capacity
Source paper
extracted_from(2024) · Minyoung Huh · Brian Cheung · Tongzhou Wang · Phillip Isola
Neighborhood — ranked by edge-count
Findings (1)
finding
- On CIFAR-10, larger models exhibit greater alignment with each other compared to smaller onessupportsKornblith et al. / Krizhevsky finding replicated in paper discussion
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Key limitation of the PRH for non-bijective observations
- Scaling model size, as well as data and task diversity, drives representational convergence toward the platonic representationhypothesis0.834Core mechanism hypothesis connecting PRH to the empirical trend of scaling in AI
- Selective pressure toward convergence via task generality
- Core framing.
- Implication of PRH for AI fairness and bias
- How do representations differ or converge between architectures, tasks, and modalities?question0.808Broader research question MAS is positioned to address, citing multiple recent works.
- Inverse scaling interpretation.
- Fig. 23.