Mutual Information Cap on Alignment

The theoretical cap on cross-modal alignment determined by mutual information between input signals and model capacity

Neighborhood — ranked by edge-count

concept

Cross-Modal Alignment
associated_with
The alignment between representations learned from different data modalities such as vision and language

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Mutual Informationconcept0.807
Expected mutual information between future states and outcomes; equivalent to intrinsic value.
Alignmentconcept0.757
The goal of making model behavior match human values and intentions, often addressed during post-training.
Mutual k-Nearest Neighbor Alignment Metricmethod0.748
Primary alignment metric used in experiments; measures mean intersection of k-nearest neighbor sets between two kernels
Data-Centric Alignmentconcept0.742
Alignment approach that focuses on curating or modifying training data; the paper bridges this with interpretability methods.
Information Gain / Mutual Informationconcept0.736
The reduction in uncertainty about hidden states afforded by observing outcomes, motivating epistemic exploration.
Mutual Embeddingconcept0.729
A reinforcing interlock between different materials, mentioned alongside Deep Interlock in West Dean construction.
Representational Alignmentconcept0.724
Measure of similarity between the similarity structures (kernels) induced by two different representations
Deliberative Alignmentframework0.721
OpenAI's approach integrating chain-of-thought reasoning into alignment; parallels contemplative self-monitoring