Topological constraints on self-organisation in locally interacting systems (Sacco, Sakthivadivel, Levin 2025)

Methods (2)

• Landau–Lifshitz scaling argument
  Historical technique for proving absence of phase transitions via free energy scaling; generalized in this paper to arbitrary local Hamiltonians
• Peierls argument
  Classical proof technique for existence of phase transitions in dimensions >1 via domain wall perimeter scaling; adapted in Theorem 1

Frameworks (2)

• graph Hamiltonian
  Hamiltonian constructed from adjacency matrix and coupling strengths on a graph
• Windowed Hamiltonian
  Core technical concept: a Hamiltonian where spin-spin interactions are defined only within finite windows ω, enabling generic analysis across diverse lattices

Findings (9)

• For a graph with independent cliques, individual cliques may flip magnetisation while remaining uniformly magnetised if intra-clique coupling > (T/2) log n_i (Theorem 4)

  Condition for hierarchical order with locally coherent but globally varying phases

• In hierarchical systems with independent cliques, there exist parameter regimes where individual cliques maintain uniform magnetisation while others flip.

  Shows how hierarchical topology enables local order within global flexibility; explains biological multiscale organization

• For one-dimensional local Hamiltonian with m>1 stored patterns at non-zero temperature, domain wall formation is thermodynamically favourable (Theorem 2)

  No ordered phase in 1D with multiple stored patterns

• All local Hamiltonians on lattices with the same combinatorial structure have asymptotically equivalent free energies (Theorem 1)

  Topological equivalence theorem for local Hamiltonians

• At thermal equilibrium, ability to converge to an ordered phase is independent of energy levels and window sizes (Lemma 1)

  Scaling argument depends only on perimeter, not details of energy magnitudes or window length

• Autoregressive model unable to converge to a single stored pattern for any finite β (Corollary 2)

  Consequence of Theorem 3 and 1D no-order result

• A unique local Hamiltonian with window length ω can be associated to any AR(ω) model (Theorem 3)

  Mapping autoregressive models to spin systems

• There exists a non-empty critical temperature range of hierarchical behaviour (Proposition 3)

  Proof that the conditions of Theorem 4 are realisable in a range of temperatures

• Causally-masked attention in a decoder-only model has no ordered phase (Proposition 2)

  Application to transformer language models

Claims (12)

• The inability for autoregressive large language models to maintain states of long-range order resembles tangential speech or derailment in formal thought disorder.

  Analogy between LLM incoherence and schizophrenia symptoms

• The results generalise readily to non-equilibrium systems where scaling relationships remain similar (e.g., dynamic or localised scaling).

  Claim about broader applicability of the scaling argument

• All intelligence is collective intelligence, in the sense that it is made of parts which must align with respect to system-level goals.

  Opening axiom of the paper, a fundamental interpretive stance

• Decoder-only transformer architectures are fundamentally limited in generating long, coherent sequences due to lack of ordered phase.

  Interpretation of Proposition 2 as a fundamental limitation on LLMs

• Practical context length limitations in language models lead to forgetting outside the window, constraining coherence over time.

  Claim about engineering constraint reinforcing the theoretical no-order result

• The difference between simple language models and multicellular organisms goes beyond the substrate of intelligence considered.

  Claim that topologies, not material substrates, account for differing organisational abilities

• Topology is the critical factor differentiating the self-organising capabilities of biological systems and language models.

  Central interpretive claim of the paper: the ability to maintain long-range order is determined by interaction topology, not substrate.

• Hierarchical structures in biological systems enable local order while globally disordered, explaining complex patterning.

  Claim that multiscale organisation produces complex patterns via clique-based local coherence

• Self-organisation can be viewed as a form of autopoietic cognition navigating problem spaces toward target morphologies.

  Linking self-organisation to cognition and navigation of configuration space

• Hierarchical structure in interaction topology enables complex multiscale patterns that cannot exist in flat networks.

  Explains why biological systems achieve organization across scales while language models struggle; grounds in free energy scaling

Hypotheses (2)

• We hypothesise this explains why stigmergy and other forms of extracellular signalling arise in biological systems, which is known to enhance the ability for a collective system to order itself.

  Hypothesis connecting fitness pressure from topological constraints to the evolutionary origin of stigmergy

• We hypothesise that an embodied world model, extending the system in space and time by its interactions with an environment, can be leveraged to maintain coherence.

  Proposed solution to the topological limitation, linking embodiment to coherence

Questions (1)

• What is the functional distinction between simple language models and multicellular organisms, and can generative AI harness that property to achieve long-range order?

  Core motivating question; drives investigation of topological differences between biological and artificial systems

Similar preprints — Semantic Scholar