paper:doi-10-1016-j-neubiorev-2016-06-022Active inference and learning
Original abstract (expand)
This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 83%
- Learning Perception and Planning with Deep Active InferenceTim Verbelen, Johannes Nauta, Cedric De Boom and Bart Dhoedt Ozan \c{C}atal2020≈ 82%
- Demonstrating the Continual Learning Capabilities and Practical Application of Discrete-Time Active InferenceRithvik Prakki2024≈ 80%
- Active inference for action-unaware agentsKeisuke Suzuki, Ryota Kanai, Manuel Baltieri Filippo Torresan2025≈ 80%
- Active Inference and Human--Computer InteractionJohn H. Williamson, Sebastian Stein Roderick Murray-Smith2024≈ 79%
- Active inference and artificial reasoningLancelot Da Costa, Alexander Tschantz, Conor Heins, Christopher Buckley, Tim Verbelen, Thomas Parr Karl Friston2025≈ 79%
- Active Inference or Control as Inference? A Unifying ViewAbraham Imohiosen, Jan Peters Joe Watson2020≈ 79%
- ≈ 79%
- Active inference: demystified and comparedPhilip J. Ball, Thomas Parr, Karl J. Friston Noor Sajid2021≈ 78%
- A Concise Mathematical Description of Active Inference in Discrete TimeCarlotta Langer, Nihat Ay Jesse van Oostrum2025≈ 78%
- On Predictive planning and counterfactual learning in active inferenceTakuya Isomura, Adeel Razi Aswin Paul2024≈ 78%
- Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action LoopChristian Guckelsberger (2), Christoph Salge (3 and 4), Sim\'on C. Smith (4 and 5), Daniel Polani (4) ((1) Araya Inc., Tokyo, Japan, (2) Computational Creativity Group, Department of Computing, Goldsmiths, University of London, London, UK, (3) Game Innovation Lab, Department of Computer Science and Engineering, New York University, New York City, NY, USA, (4) Sepia Lab, Adaptive Systems Research Group, Department of Computer Science, University of Hertfordshire, Hatfield, UK, (5) Institute of Perception, Action and Behaviour, School of Informatics, The University of Edinburgh, UK) Martin Biehl (1)2018≈ 78%
- ≈ 77%
- Active Inference in Robotics and Artificial Agents: Survey and ChallengesCristian Meo, Corrado Pezzato, Ajith Anil Meera, Mohamed Baioumy, Wataru Ohata, Alexander Tschantz, Beren Millidge, Martijn Wisse, Christopher L. Buckley, Jun Tani Pablo Lanillos2021≈ 77%
- Active Inference on the Edge: A Design StudyVictor Casamayor Pujol, Praveen Kumar Donta, Schahram Dustdar Boris Sedlak2023≈ 77%
- Active inference: demystified and comparedin corpus2021≈ 76%
- ≈ 71%
- Active Inference: A Process Theoryin corpus2017≈ 71%
- ≈ 70%
- Active Inference, Curiosity and Insightin corpus2017≈ 68%
- ≈ 65%
- ≈ 65%
- Learning without neurons in physical systemsin corpus2022≈ 63%
- ≈ 63%
- ≈ 63%
- ≈ 63%
- ≈ 62%
- Why Learning Requires Feelingin corpus2026≈ 62%
- ≈ 62%
- Interpreting Language Model Parametersin corpus2026≈ 62%
Similar preprints — Semantic Scholar
Cited by (6)
- Active Inference with a Self-Prior in the Mirror-Mark Task
Spontaneous mark-directed behavior in the mirror-mark task emerges from a single internal mechanism—the self-prior—combined with expected free energy minimization, without any external reward signal.
- Active Inference, Curiosity and Insight
Minimizing expected variational free energy under a discrete-state Markov decision process generative model is sufficient to produce curiosity, epistemic learning, and insight without any additional m
- Active inference: demystified and compared
Active inference agents operating under expected free energy minimization achieve 98.90 [98.00, 99.79] average score in a non-stationary FrozenLake OpenAI gym environment, compared to 64.39 [60.33, 68
- Active Inference: A Process Theory
A single variational principle—minimizing variational free energy via gradient descent on a Markov decision process (MDP) generative model—is sufficient to derive neuronal dynamics that reproduce, wit
- Active inference on discrete state-spaces: a synthesis
Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, plannin
- World models, artificial general intelligence and the hard problems of life–mind continuity: toward a unified understanding of natural and artificial intelligence