Cortex as a Transformer

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Transformer can be viewed as a Wolfram causal graph with foliations specifying computation order.claim0.764
  Janus's interpretive framing of transformers as causal graphs.
• Transformer Neural Networkframework0.757
  Core machine learning architecture analyzed in the paper; shown to be mathematically related to TEM.
• R-Transformer: Recurrent Neural Network Enhanced Transformer (Wang et al., 2019)concept0.750
  Prior work on recurrently generated position encodings; cited as precedent for TEM-t's recurrent position encoding method.
• A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)concept0.735
  Foundational mechanistic interpretability paper on transformer circuit analysis
• what takes the role of memory neurons if not hippocampus in cortical transformer implementations?question0.733
  Open question about cortical instantiation of transformer-like memory when hippocampus is not involved.
• Decision Transformermethod0.730
  A model that frames RL as sequence modeling, SOTA from random trajectories.
• Zero-Layer Transformerconcept0.728
  A transformer with no attention layers; shown to model bigram statistics via T = W_U W_E
• transformer architectureframework0.727
  Neural network architecture based on attention, commonly used in large language models