what is the analogue of spatial positional encodings for higher order tasks such as language?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Positional encodings inferred on the fly from previously learned structures would offer fruitful research direction for language, maths, and logicclaim0.812
  Forward-looking interpretive claim about the implications of recurrent position encodings for NLP research.
• Position Encodingsconcept0.796
  Mechanism for encoding sequence order in transformers; paper argues these should reflect learned structural representations rather than fixed sines/cosines.
• Position encodings should represent location in a learned structure inferred on the fly rather than fixed sines and cosinesclaim0.771
  Novel interpretive claim about position encodings inspired by the TEM-transformer correspondence.
• Recurrent Position Encodingsmethod0.764
  Key modification to transformers proposed in this paper: position encodings generated by a recurrent network trained on action sequences.
• A diagram spatializes semantic value using graphic features of spatial organization to express relationsclaim0.755
• An interplay between causal abstraction and feature geometry deepens mechanistic understanding of language modelsclaim0.748
  Methodological claim about the scientific value of combining causal abstraction with representational geometry analysis
• A General Language Assistant as a Laboratory for Alignment (Askell et al. 2021)concept0.742
  HHH training framework that Claude was trained with prior to experiments
• Spatial Reasoningconcept0.733
  The ability to reason about shapes, space, and topology, essential for ancient mathematical discoveries and observed in many animals.