Karl J. Friston

Authored papers (2)

• Active inference: demystified and compared2021ⓒ 135

  Active inference agents operating under expected free energy minimization achieve 98.90 [98.00, 99.79] average score in a non-stationary FrozenLake OpenAI gym environment, compared to 64.39 [60.33, 68.44] for Bayesian model-based RL with Thompson sampling and 66.08 [63.28, 68.88] for Q-learning (ε=0.1) — a performance gap that emerges specifically because active inference treats environmental change as a context-inference problem rather than a reversal-learning problem, recovering within a single episode after each goal-hole swap. The paper introduces a discrete state-space and time formulation of active inference as its primary expository instrument, decomposing expected free energy G into an epistemic value term (mutual information between outcomes and hidden states) and an extrinsic value term (KL divergence between predicted and preferred outcomes), showing that both exploration and exploitation are expressions of a single objective rather than requiring separate engineering via ε-greedy schedules or temperature hyperparameters. In reward-free conditions where Q-learning freezes into a deterministic circular policy scoring 0.00, the active inference null model (zero prior preferences) still scores 50.03 [49.70, 50.35] through pure information-seeking, and agents equipped with Dirichlet hyperpriors over outcome preferences learn stable behavioral niches — including counter-intuitive hole-seeking — without any external reward signal. The paper argues this implies that reinforcement learning is a limiting special case of active inference in which the epistemic value term is suppressed and preferences are fixed externally, and that reward-free preference learning dissolves the circularity of the reward hypothesis rather than merely circumventing it.

• Active Inference, Curiosity and Insight2017ⓒ 377

  Minimizing expected variational free energy under a discrete-state Markov decision process generative model is sufficient to produce curiosity, epistemic learning, and insight without any additional machinery. Friston et al. 2017 demonstrates this across two linked mechanisms: first, including posterior beliefs about likelihood parameters **A** in expected free energy G(π) introduces a novelty term—information gain about model parameters—that drives agents to sample combinations of hidden states and outcomes they have not yet encountered, resolving ignorance rather than merely ambiguity or risk. Second, Bayesian model reduction (implemented via the spm_MDP_VB_X.m routine in SPM) allows post-hoc or online pruning of redundant concentration parameters: a reduced model is accepted when ΔF ≤ −3, corresponding to a Bayes factor of approximately 20:1 in favor of the simpler model. Simulated agents learning a 3-rule, 4-factor abstract contingency task (144 hidden-state combinations, 36 possible outcomes) reach near-perfect performance after roughly 14 trials under pure epistemic learning, dropping to approximately 10 trials when online Bayesian model reduction is applied across 64 simulated agents. The sleep analog—non-REM synaptic pruning followed by REM-like belief re-evaluation—is formalized identically through the same free energy difference equation. The paper argues this implies that aha moments are necessarily subpersonal events (optimization of the generative model itself, not modeling of that optimization), that the quality of intelligence is inversely related to the thermodynamic energy expended during convergence via the Jarzynski equality, and that communicating reduced model priors rather than parameter posteriors constitutes a principled formal account of shared knowledge—consciousness in the pre-Cartesian sense of con-scire.

More papers — OpenAlex / S2

Their work is cited by (1)

• Active inference on discrete state-spaces: a synthesis1× refs

Recent mentions (3)

• papers

  friston-2017-active.md
• papers-typed

  Multiple ways to implement and infer sentience.md
• papers-typed

  sajid_2021_active_inference_demystified.md