finding

active

finding:in-the-absence-of-prior-preferences-active-inference-null-model-and-bayesian-rl-maintain-exploration-with-average-scores-of-44-00-and-39-94-respectively-whereas-q-learning-does-not-explore

In the absence of prior preferences, Active Inference null model and Bayesian RL maintain exploration with average scores of 44.00 and 39.94 respectively, whereas Q-learning does not explore.

Table 2 first row; reward shaping section.

Source paper

extracted_from

Active inference: demystified and compared

(2021) · Noor Sajid · Philip J. Ball · Thomas Parr · Karl J. Friston

Neighborhood — ranked by edge-count

Questions (1)

question

How does active inference compare to reinforcement learning in environments with no rewards or uninformative prior preferences?
answered_by
Core question addressed by the simulations when rewards are removed.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Under reward shaping (G=100, H=-100, F=0), Active Inference scored 99.52, Bayesian RL 99.77, Q-learning 95.56, with nearly identical behavior between belief-based agents.finding0.862
Table 2, row 3, showing equivalence when prior preferences match rewards.
Active Inference null model (no prior preferences) achieved average score 50.03 [49.70, 50.35] in deterministic FrozenLake.finding0.859
Table 1.
Active inference agents engage in information-seeking behavior in reward-free FrozenLake environments, contrasting with Q-learning but similar to Bayesian RL.finding0.812
Empirical demonstration on FrozenLake; shows epistemic value drives exploration absent reward signal.
There is an implicit behavioral equivalence between Bayesian model-based reinforcement learning and active inference when prior preferences are treated as a reward function.claim0.810
§3, reward shaping conclusion.
All three agent types (active inference, Q-learning, Bayesian RL) perform adequately in stationary FrozenLake; only active inference achieves Bayes-optimal behavior in non-stationary settings.finding0.809
Key empirical result validating online planning capability of active inference.
Active inference recovers performance within 1 episode after context switch in non-stationary FrozenLake, while Bayesian RL requires ~40 episodes.finding0.808
Figure 4 and discussion in §3.
Active inference and Bayesian model-based RL learn reward-maximizing behavior in <10 episodes in deterministic FrozenLake.finding0.796
Discussion of Figure 3.
Human participants in the rule-learning paradigm should acquire insight after approximately 7–8 trials, fewer than the ~14 required by Bayes-optimal inference without model reduction, suggesting they perform Bayesian model selection.hypothesis0.794
Based on informal audience experiments; implies people use prior knowledge about rule structure