concept
active
concept:information-bottleneck-in-workspaceInformation bottleneck in workspace
Limited capacity of the workspace relative to the sum of module capacities.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (2)
concept
- Information Bottleneckrelated_toCompression-prediction trade-off; NIS encodes micro-states through an information bottleneck.
- GWT-2: Limited capacity workspace, entailing a bottleneck and a selective attention mechanismassociated_withIndicator: workspace bottleneck with selective attention.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- A lower-dimensional activation that is the only pathway for information between higher-dimensional activations; e.g. the residual stream between MLP layers
- Architectural feature integrating information from modules and broadcasting it, associated with consciousness per GWT.
- Point of genuine disagreement with IIT and GWT
- Failure mode for output-surjectivity: LLMs may lack capacity to predict all tokens due to rank constraints
- The interior domain of the text, the space of inscription.
- Population structure mechanism implementing genetic assortment; cited as example of individuation mechanism in multicellularity.
- The exterior domain outside the text block, the surrounding world.