finding

active

finding:active-inference-agents-engage-in-information-seeking-behavior-in-reward-free-frozenlake-environments-contrasting-with-q-learning-but-similar-to-bayesian-rl

Active inference agents engage in information-seeking behavior in reward-free FrozenLake environments, contrasting with Q-learning but similar to Bayesian RL.

Empirical demonstration on FrozenLake; shows epistemic value drives exploration absent reward signal.

Source paper

extracted_from

Active inference: demystified and compared

(2021) · Noor Sajid · Philip J. Ball · Thomas Parr · Karl J. Friston

Neighborhood — ranked by edge-count

Claims (1)

claim

Active inference agents can learn their own reward function (prior preferences) by interacting with the environment, bypassing the need for an explicit reward signal.
supports
Abstract and §3, preference learning section.

Communities (3)

community

Active inference & agent ecology
members_of
Free energy minimization, Markov blankets, trust gradients, and multi-agent rhythm/deferral frameworks
Active inference & free energy minimization
members_of
Friston's framework unifying perception, action, and learning under variational free energy minimization.
Active inference as Bayes-optimal exploration
members_of
Compares active inference to Q-learning and Bayesian RL across stationary and non-stationary environments, emphasizing information-seeking behavior and theoretical optimality.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

All three agent types (active inference, Q-learning, Bayesian RL) perform adequately in stationary FrozenLake; only active inference achieves Bayes-optimal behavior in non-stationary settings.finding0.899
Key empirical result validating online planning capability of active inference.
Active inference and Bayesian model-based RL learn reward-maximizing behavior in <10 episodes in deterministic FrozenLake.finding0.875
Discussion of Figure 3.
Active inference recovers performance within 1 episode after context switch in non-stationary FrozenLake, while Bayesian RL requires ~40 episodes.finding0.839
Figure 4 and discussion in §3.
Active inference achieves Bayes-optimal behavior in non-stationary environments through online belief updating.hypothesis0.837
Tested via FrozenLake experiments; predicts superior performance when environment dynamics change.
Under reward shaping (G=100, H=-100, F=0), Active Inference scored 99.52, Bayesian RL 99.77, Q-learning 95.56, with nearly identical behavior between belief-based agents.finding0.836
Table 2, row 3, showing equivalence when prior preferences match rewards.
How does active inference compare to reinforcement learning in environments with no rewards or uninformative prior preferences?question0.822
Core question addressed by the simulations when rewards are removed.
Active inference agents can learn epistemic habits in the absence of extrinsic rewards through ambiguity minimization.claim0.822
§2, summarizing information-seeking behavior.
Active inference offers an attractive natural adaptation mechanism for non-stationary environments due to its Bayesian model updating properties.claim0.819
§3, after non-stationary results.