Zhang et al. 2023: STORM: Efficient Stochastic Transformer based World Models (NeurIPS 2023)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Sauers' statistical anomaly: when models are given Janus post explaining transformers, reconstruction accuracy tails extend both ways, with ~1/1000 reconstructions anomalously accuratefinding0.722
  Statistically rigorous analysis of Claude introspection; suggests models may have latent introspective capabilities that can be enhanced or disrupted.
• Stochastic text generation (next token prediction)method0.716
  The core mechanism of LLMs: predicting the next token based on previous context.
• Friston & Penny (2011) — Post hoc Bayesian model selectionconcept0.710
  Source paper for Bayesian model reduction methodology used in structure learning
• Small-scale looped transformers trained from scratch with constant 4-recurrence schedule and simplified loss self-organize into multiple distinct mixing stages mirroring feedforward modelsfinding0.710
  Evidence that stages of inference emerge without training biases from retrofitting, recurrence scheduling, or multi-recurrence losses
• What are effective tools and modelling paradigms for implementing and running (simulations of) self-modifying open systems?question0.709
  Organizing question for §6.2 on computational tools.
• Transformers are recurrent through autoregression because K/V stream provides horizontal information flow across positions.claim0.709
  Claim formalizing the Anima Labs idea that transformers are effectively recurrent due to K/V stream.
• Power's ecosystem model with individual-level selection on interaction traits evolves to solve Sudoku puzzles (Power 2019)finding0.708
  Demonstrates that ecological networks can learn complex problem-solving without system-level selection
• The Guasare simulation showed that the defined process generates coherent, complex, and variable morphology for houses, streets, lots and gardens even without influence from external factors such as land and topographyfinding0.707
  Key validation that the process itself — not just site conditions — generates living structure.