Superposition exploits the geometry of high-dimensional spaces, which allow exponentially many almost-orthogonal vectors but only n strictly orthogonal ones.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.815
  Explanation for why dictionary learning can recover many more features than dimensions.
• "We could continue to invent new and exciting mechanisms beyond context-orientation and Worlds, ad absurdum, or recognise they are all just points within a dimensionally unbounded space of related, self-similar mechanisms."concept0.764
  Claim that many advanced programming paradigms reduce to parameterizations of the n-way associative model.
• The deep nature of space appears to be so profound that just the geometric unfolding, when done properly, leads to results which make sense from an engineering point of view.claim0.762
  The mystery that beautiful geometry often yields good structural behavior is acknowledged but not yet fully explained mathematically.
• DIM captures only one facet of the multi-dimensional truth subspace; additional orthogonal structure exists beyond itclaim0.761
  Interpretation of Experiment 4 cosine similarity results
• The wholeness of geometry is intensified and brought to its maximum potential by inner light in color.claim0.755
• The two-dimensional subspace reported by Burger et al. (2024) seems to reflect a stage of transition in the model's processing, rather than a universal property of truth directions.quote0.751
  Load-bearing interpretive claim about the layer-specificity of Burger et al.'s finding.
• The goodness of the results is visible in the evident life and in the profound fact that unfolding generates an 'architecture'.claim0.748
  The dual validation of living process: life and the emergence of architectural order.
• Does the multi-directional nature of truth imply an underlying nonlinear representation, or is it compatible with linear separability?question0.747
  Theoretical open question about the geometry of truth in LLMs raised in Discussion