Local vs Compositional Representations

Theoretical distinction between representing token-context pairs as individual features (local) vs combining independent features (compositional)

Neighborhood — ranked by edge-count

concept

Token-in-Context Feature
associated_with
Feature that fires on a specific token only within a specific surrounding context (e.g., 'the' in physics vs 'the' in mathematics)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Localist Representationsconcept0.814
Prior assumption that high-level variables align with disjoint groups of neurons in standard basis; contrasted with distributed representations.
Linear representationconcept0.732
The idea that features are encoded as directions in activation space.
Structure in representationsconcept0.731
The central question of whether representational geometry implies corresponding computational structure
Higher-Order Representationsconcept0.730
Representations of one's own mental states; associated with consciousness in higher-order theories.
How do representations differ or converge between architectures, tasks, and modalities?question0.730
Broader research question MAS is positioned to address, citing multiple recent works.
Compositional Reasoningconcept0.728
Modular reasoning enabled by monoid and other algebraic laws; central to maintainability and correctness of functional systems.
What is compositionality?question0.726
Primary guiding question for the paper; explores multiple formulations and uses of the term.
Deep Representationsconcept0.726
Hierarchical representations in neural networks that allow compression and coordinated behaviour while retaining sensitivity to input changes.