Learning to encode position for transformer with continuous dynamical model (Liu et al., 2020)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Transformers develop self-models through in-context learning, not just training data; even old base models without LLM-related text can bootstrap self-referential reasoning at runtime.claim0.794
  Antra's foundational claim about how introspection arises computationally rather than from memorised text.
• Transformers learn in-context by gradient descent, functioning as mesa-optimizers that learn internal models in real timefinding0.793
  Evidence that in-context learning is not mere pattern matching but genuine optimization, relevant to applying the thesis to inference
• A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)concept0.769
  Foundational mechanistic interpretability paper on transformer circuit analysis
• Position encodings should represent location in a learned structure inferred on the fly rather than fixed sines and cosinesclaim0.760
  Novel interpretive claim about position encodings inspired by the TEM-transformer correspondence.
• Transformer cognition operates via interference patterns across redundant paths, encoding nuanced state deltas similar to continuous human memory.claim0.757
  Proposes transformers experience cognition as interference-based and continuous; connects to Anima Labs reports of parallel processing.
• The transformer likely uses a local code for token-in-context features rather than purely compositional representations, because local codes enable sharper predictionsclaim0.756
  Authors argue the prevalence of token-in-context features reflects genuine model computation rather than dictionary learning artifact
• TEM's path-integration representation g plays the role of position encodings in transformersclaim0.756
  Key structural correspondence claim linking the neuroscience model's spatial representation to ML concept of position encoding.
• In-Context Learning of Representations (Park et al. 2025)framework0.754
  Reports phase-like breakpoints and geometry changes as context scales; UCCT provides measurable predictor