Simulators (LessWrong post)

A foundational variational principle from statistical physics that formalizes how self-organizing systems maintain structural integrity and adapt to their environment by minimizing free energy—a mathematical bound on surprise or prediction error. Originally developed by Karl Friston, the framework unifies action, perception, and learning as processes of active inference, where systems both update internal models of the world and act upon it to reduce the divergence between predictions and observations.

Neural network architecture based on attention, commonly used in large language models

The framework proposed: self-supervised models are simulators that generate simulacra; distinguishes simulator from simulated agents.

Bostrom's category of AIs that produce desired results given commands but do not act autonomously.

The view of AI as a question-answer system optimized for correctness, often inherited from supervised learning.

Bostrom's category of AIs that perform specific tasks without overarching goals.

The traditional alignment framework focusing on agents optimized to pursue goals.

The approach of learning from demonstrations, often assuming a single agent; Paul Christiano used 'mimicry'.

The property of an AI being safe to shut down or modify; discussed in context of GPT.

The concept of inner vs outer alignment, referenced multiple times.

The thesis that sufficiently advanced agents will converge to similar subgoals.

Classical thesis that any level of intelligence can be combined with any goal.

A framework for agents that are part of the environment they act on.

A theory about inner misalignment mentioned in footnote.