paper:doi-10-1080-17588928-2015-1020053Active inference and epistemic value
Original abstract (expand)
We offer a formal treatment of choice behavior based on the premise that agents minimize the expected free energy of future outcomes. Crucially, the negative free energy or quality of a policy can be decomposed into extrinsic and epistemic (or intrinsic) value. Minimizing expected free energy is therefore equivalent to maximizing extrinsic value or expected utility (defined in terms of prior preferences or goals), while maximizing information gain or intrinsic value (or reducing uncertainty about the causes of valuable outcomes). The resulting scheme resolves the exploration-exploitation dilemma: Epistemic value is maximized until there is no further information gain, after which exploitation is assured through maximization of extrinsic value. This is formally consistent with the Infomax principle, generalizing formulations of active vision based upon salience (Bayesian surprise) and optimal decisions based on expected utility and risk-sensitive (Kullback-Leibler) control. Furthermore, as with previous active inference formulations of discrete (Markovian) problems, ad hoc softmax parameters become the expected (Bayes-optimal) precision of beliefs about, or confidence in, policies. This article focuses on the basic theory, illustrating the ideas with simulations. A key aspect of these simulations is the similarity between precision updates and dopaminergic discharges observed in conditioning paradigms.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 78%
- ≈ 78%
- Active inference and artificial reasoningLancelot Da Costa, Alexander Tschantz, Conor Heins, Christopher Buckley, Tim Verbelen, Thomas Parr Karl Friston2025≈ 76%
- Active Inference or Control as Inference? A Unifying ViewAbraham Imohiosen, Jan Peters Joe Watson2020≈ 76%
- Learning Perception and Planning with Deep Active InferenceTim Verbelen, Johannes Nauta, Cedric De Boom and Bart Dhoedt Ozan \c{C}atal2020≈ 76%
- Active inference, Bayesian optimal design, and expected utilityLancelot Da Costa, Thomas Parr, Karl Friston Noor Sajid2021≈ 76%
- Active Inference and Human--Computer InteractionJohn H. Williamson, Sebastian Stein Roderick Murray-Smith2024≈ 76%
- ≈ 75%
- ≈ 75%
- Demonstrating the Continual Learning Capabilities and Practical Application of Discrete-Time Active InferenceRithvik Prakki2024≈ 75%
- Active inference: demystified and comparedPhilip J. Ball, Thomas Parr, Karl J. Friston Noor Sajid2021≈ 75%
- Active inference for action-unaware agentsKeisuke Suzuki, Ryota Kanai, Manuel Baltieri Filippo Torresan2025≈ 75%
- ≈ 75%
- Prior Preference Learning from Experts:Designing a Reward with Active InferenceCheolhyeong Kim, Hyung Ju Hwang Jin young Shin2021≈ 74%
- A Concise Mathematical Description of Active Inference in Discrete TimeCarlotta Langer, Nihat Ay Jesse van Oostrum2025≈ 74%
- Active inference: demystified and comparedin corpus2021≈ 74%
- ≈ 69%
- ≈ 69%
- Active Inference: A Process Theoryin corpus2017≈ 69%
- Active Inference, Curiosity and Insightin corpus2017≈ 68%
- ≈ 63%
- ≈ 62%
- ≈ 62%
- Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiesin corpus2023≈ 62%
- Cognitive glues are shared models of relative scarcities: the economics of collective intelligencein corpus2026≈ 62%
- ≈ 61%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 61%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 61%
- When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Modelsin corpus2025≈ 61%
- Information, Processes and Gamesin corpus≈ 61%
Similar preprints — Semantic Scholar
Cited by (5)
- Active Inference, Curiosity and Insight
Minimizing expected variational free energy under a discrete-state Markov decision process generative model is sufficient to produce curiosity, epistemic learning, and insight without any additional m
- Active inference: demystified and compared
Active inference agents operating under expected free energy minimization achieve 98.90 [98.00, 99.79] average score in a non-stationary FrozenLake OpenAI gym environment, compared to 64.39 [60.33, 68
- Active Inference: A Process Theory
A single variational principle—minimizing variational free energy via gradient descent on a Markov decision process (MDP) generative model—is sufficient to derive neuronal dynamics that reproduce, wit
- Technological Approach to Mind Everywhere: An Experimentally-Grounded Framework for Understanding Diverse Bodies and Minds
TAME—Technological Approach to Mind Everywhere—formalizes a non-binary, empirically grounded framework for recognizing, comparing, and manipulating cognition across radically diverse substrates, from
- Active inference on discrete state-spaces: a synthesis
Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, plannin