finding

active

finding:all-three-agent-types-active-inference-q-learning-bayesian-rl-perform-adequately-in-stationary-frozenlake-only-active-inference-achieves-bayes-optimal-behavior-in-non-stationary-settings

All three agent types (active inference, Q-learning, Bayesian RL) perform adequately in stationary FrozenLake; only active inference achieves Bayes-optimal behavior in non-stationary settings.

Key empirical result validating online planning capability of active inference.

Source paper

extracted_from

Active inference: demystified and compared

(2021) · Noor Sajid · Philip J. Ball · Thomas Parr · Karl J. Friston

Neighborhood — ranked by edge-count

Hypotheses (1)

hypothesis

Active inference achieves Bayes-optimal behavior in non-stationary environments through online belief updating.
answered_by
Tested via FrozenLake experiments; predicts superior performance when environment dynamics change.

Communities (3)

community

Active inference & agent ecology
members_of
Free energy minimization, Markov blankets, trust gradients, and multi-agent rhythm/deferral frameworks
Active inference & free energy minimization
members_of
Friston's framework unifying perception, action, and learning under variational free energy minimization.
Active inference as Bayes-optimal exploration
members_of
Compares active inference to Q-learning and Bayesian RL across stationary and non-stationary environments, emphasizing information-seeking behavior and theoretical optimality.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Active inference agents engage in information-seeking behavior in reward-free FrozenLake environments, contrasting with Q-learning but similar to Bayesian RL.finding0.899
Empirical demonstration on FrozenLake; shows epistemic value drives exploration absent reward signal.
Active inference and Bayesian model-based RL learn reward-maximizing behavior in <10 episodes in deterministic FrozenLake.finding0.867
Discussion of Figure 3.
Active inference recovers performance within 1 episode after context switch in non-stationary FrozenLake, while Bayesian RL requires ~40 episodes.finding0.865
Figure 4 and discussion in §3.
Active inference offers an attractive natural adaptation mechanism for non-stationary environments due to its Bayesian model updating properties.claim0.821
§3, after non-stationary results.
Active Inference agent achieved average score 99.88 [99.64, 100.00] in deterministic FrozenLake environment across 200 trials of 500 episodes.finding0.816
Table 1, deterministic environment row.
In the absence of prior preferences, Active Inference null model and Bayesian RL maintain exploration with average scores of 44.00 and 39.94 respectively, whereas Q-learning does not explore.finding0.809
Table 2 first row; reward shaping section.
Under reward shaping (G=100, H=-100, F=0), Active Inference scored 99.52, Bayesian RL 99.77, Q-learning 95.56, with nearly identical behavior between belief-based agents.finding0.808
Table 2, row 3, showing equivalence when prior preferences match rewards.
Active inference agents can learn their own reward function (prior preferences) by interacting with the environment, bypassing the need for an explicit reward signal.claim0.807
Abstract and §3, preference learning section.