TEM-t learns band-cell-like position encoding representations resembling Krupic et al. band cells

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• TEM-t with linear activations learns grid-cell-like position encoding representations in 2D spatial environmentsfinding0.856
  Empirical result showing TEM-t recapitulates entorhinal grid cell representations with linear post-transition activation.
• TEM-t learns grid cells in hexagonal 6-connected worldsfinding0.793
  Empirical extension showing grid cell learning generalises to non-4-connected spatial environments.
• Position encodings should represent location in a learned structure inferred on the fly rather than fixed sines and cosinesclaim0.787
  Novel interpretive claim about position encodings inspired by the TEM-transformer correspondence.
• TEM-t memory neurons show spatially-tuned firing resembling hippocampal place cells in each environmentfinding0.780
  Empirical result demonstrating that the sparse softmax activation of memory neurons produces place-cell-like spatial tuning.
• TEM-t instantiates hippocampal indexing theory by using memory neurons to bind cortical representations across brain regionsclaim0.776
  Theoretical claim linking the TEM-t architecture to the Teyler-Rudy hippocampal indexing theory.
• Neural Representations of Location Composed of Spatially Periodic Bands (Krupic et al., 2012)concept0.775
  Discovery of band cells; TEM-t also recapitulates these representations.
• TEM's path-integration representation g plays the role of position encodings in transformersclaim0.759
  Key structural correspondence claim linking the neuroscience model's spatial representation to ML concept of position encoding.
• Position Encodingsconcept0.753
  Mechanism for encoding sequence order in transformers; paper argues these should reflect learned structural representations rather than fixed sines/cosines.