claim
active
claim:different-models-cannot-converge-to-the-same-representation-if-they-have-access-to-fundamentally-different-information-convergence-is-capped-by-mutual-information-between-input-signalsDifferent models cannot converge to the same representation if they have access to fundamentally different information; convergence is capped by mutual information between input signals
Key limitation of the PRH for non-bijective observations
Source paper
extracted_from(2024) · Minyoung Huh · Brian Cheung · Tongzhou Wang · Phillip Isola
Neighborhood — ranked by edge-count
Findings (1)
finding
- Tests information-level cap on cross-modal alignment
Concepts (1)
concept
- Bijective Observation FunctionsupportsThe idealized assumption that observations are bijective mappings of events, required for cross-modal convergence proof
Quotes (1)
quote
- The paper's central thesis statement, presented prominently after the abstract
Questions (1)
question
- Counterexample question about modality-specific information limits
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.853Selective pressure toward convergence via model capacity
- How do representations differ or converge between architectures, tasks, and modalities?question0.833Broader research question MAS is positioned to address, citing multiple recent works.
- Scaling model size, as well as data and task diversity, drives representational convergence toward the platonic representationhypothesis0.818Core mechanism hypothesis connecting PRH to the empirical trend of scaling in AI
- The central hypothesis of the paper; the platonic representation hypothesis itself
- Supported by the geometric transition visible in cosine similarity heatmaps for F0-F3.
- Empirical evidence for the universality hypothesis cited as supporting the possibility of convergent consciousness-like solutions
- Authors note robotics lacks a standardized representation approach and sufficient training data diversity to show PRH effects