Information Bottleneck

Compression-prediction trade-off; NIS encodes micro-states through an information bottleneck.

Neighborhood — ranked by edge-count

framework

Neural Information Squeezer (NIS)
implements
Machine learning framework for causal emergence identification via encoder-dynamics learner-decoder architecture.

concept

Information bottleneck in workspace
related_to
Limited capacity of the workspace relative to the sum of module capacities.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Bottleneck Activationconcept0.779
A lower-dimensional activation that is the only pathway for information between higher-dimensional activations; e.g. the residual stream between MLP layers
Developmental Bottleneckconcept0.768
Population structure mechanism implementing genetic assortment; cited as example of individuation mechanism in multicellularity.
Value of Informationconcept0.764
Economic concept related to epistemic value in expected free energy; information needed to realize rewards
Softmax Bottleneckconcept0.759
Failure mode for output-surjectivity: LLMs may lack capacity to predict all tokens due to rank constraints
conservation of information flowconcept0.752
Logic seen as conservation laws for information flow; proofs correspond to generalized copy-cat strategies.
Information Dynamicsframework0.739
Proposed theoretical framework combining qualitative and quantitative aspects of information, with explicit treatment of processes and information flow; central organizing concept for the paper.
imperfect informationconcept0.736
Game condition where players do not know the exact money values held by opponents, only counts.
Redundant Informationconcept0.735
Information shared across all sources in PID.