thinker
active
thinker:noor-sajid

Noor Sajid

name_hash f700e898b6d0f7664578ff33…
Authored
2
Introduces
0
Studies
0
Affiliations
1
Cited by
0

Authored papers (2)

  • Active inference: demystified and compared2021ⓒ 135

    Active inference agents operating under expected free energy minimization achieve 98.90 [98.00, 99.79] average score in a non-stationary FrozenLake OpenAI gym environment, compared to 64.39 [60.33, 68.44] for Bayesian model-based RL with Thompson sampling and 66.08 [63.28, 68.88] for Q-learning (ε=0.1) — a performance gap that emerges specifically because active inference treats environmental change as a context-inference problem rather than a reversal-learning problem, recovering within a single episode after each goal-hole swap. The paper introduces a discrete state-space and time formulation of active inference as its primary expository instrument, decomposing expected free energy G into an epistemic value term (mutual information between outcomes and hidden states) and an extrinsic value term (KL divergence between predicted and preferred outcomes), showing that both exploration and exploitation are expressions of a single objective rather than requiring separate engineering via ε-greedy schedules or temperature hyperparameters. In reward-free conditions where Q-learning freezes into a deterministic circular policy scoring 0.00, the active inference null model (zero prior preferences) still scores 50.03 [49.70, 50.35] through pure information-seeking, and agents equipped with Dirichlet hyperpriors over outcome preferences learn stable behavioral niches — including counter-intuitive hole-seeking — without any external reward signal. The paper argues this implies that reinforcement learning is a limiting special case of active inference in which the epistemic value term is suppressed and preferences are fixed externally, and that reward-free preference learning dissolves the circularity of the reward hypothesis rather than merely circumventing it.

  • Active inference on discrete state-spaces: a synthesis2020ⓒ 212

    Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, planning, decision-making, learning, and structure learning under two objective functions: variational free energy (an upper bound on surprise minimized during state estimation) and expected free energy G(π) (minimized during policy selection). The synthesis derives neuronal dynamics from first principles via gradient descent on free energy, showing that state estimation corresponds to a softmax function of accumulated prediction errors—equations interpretable as membrane potentials mapping to firing rates—and that these dynamics coincide exactly with variational message passing, while the Bethe approximation yields belief propagation. Policy selection follows Q(π) = σ(−G(π)), where G decomposes into risk (KL divergence between predicted and preferred states) and ambiguity (expected entropy of outcomes given states), formally subsuming KL control, expected utility theory, and optimal Bayesian design as special cases. Learning of A follows Dirichlet parameter accumulation **a** = a + Σ(oτ ⊗ sτ), which is formally equivalent to Hebbian plasticity, while structure learning proceeds via Bayesian model reduction (BMR) for simplification and Bayesian model expansion for concept acquisition, with the marginal approximation implemented in `spm_MDP_VB_X.m` identified as the most biologically plausible free energy approximation. The paper argues this implies that biological cognition—from saccadic sampling at ~4 Hz to dopaminergic precision encoding γ—is fully accountable as free energy minimization, and that the outstanding challenge is identifying the evidence-maximizing generative model an agent actually employs, which would constitute a complete structure learning roadmap.

More papers — OpenAlex / S2

Affiliations (1)

Co-authors (7)

Recent mentions (4)