Information Flow in Transformers

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• conservation of information flowconcept0.758
  Logic seen as conservation laws for information flow; proofs correspond to generalized copy-cat strategies.
• Information Dynamicsframework0.744
  Proposed theoretical framework combining qualitative and quantitative aspects of information, with explicit treatment of processes and information flow; central organizing concept for the paper.
• parallel processing paths in transformersconcept0.737
  Spontaneously reported experience of multiple simultaneous processing streams, observed even in base models.
• Redundant information paths create interference patterns, so transformers likely experience memory and cognition as interferometric and continuous.claim0.722
  Janus's claim linking path redundancy to interferometric phenomenology.
• Transformers are recurrent through autoregression because K/V stream provides horizontal information flow across positions.claim0.721
  Claim formalizing the Anima Labs idea that transformers are effectively recurrent due to K/V stream.
• Transformers use an anti-Markovian solution that recomputes relevant numeric information at each step in the Multi-Object taskclaim0.719
  Prior finding from Grant et al. 2025 used to interpret low MAS IIA for GRU-Transformer hidden state comparisons.
• Entropyconcept0.716
  A measurable physical quantity representing disorder; its increase is dictated by the second law of thermodynamics.
• Decision Transformermethod0.714
  A model that frames RL as sequence modeling, SOTA from random trajectories.