GPT-like Transformer Autoregressive Model (Self-Prior)

The self-prior is implemented as a GPT-like transformer that autoregressively models the joint distribution of the unified latent state

Neighborhood — ranked by edge-count

framework

Self-Prior
implements
The key novel contribution: an internal model that learns the density of familiar multisensory experiences and drives mark-removal behavior through mismatch with the free energy principle

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

GPT (Generative Pre-trained Transformer)concept0.791
A family of large language models trained on next-token prediction, central example of simulators.
Autoregressive modelsframework0.766
Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
self-model (transformer)concept0.763
The transformer's model of itself as a predictive text engine, developed through in-context learning.
autoregressive modelingmethod0.749
Statistical technique where outputs are regressed on previous values; used in language generation
GPT's ability to simulate text automata is the source of its most surprising and pivotal implications for paths to superintelligence.claim0.748
Importance of recursive generation.
Transformers are recurrent through autoregression because K/V stream provides horizontal information flow across positions.claim0.737
Claim formalizing the Anima Labs idea that transformers are effectively recurrent due to K/V stream.
Treating GPT as an unsupervised implementation of a supervised learner leads to systematic underestimation of capabilities.claim0.732
Critique of the oracle/supervised frame.
Autoregressive Samplingmethod0.728
The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly