paper:doi-10-1016-j-neuroimage-2011-03-062Post hoc Bayesian model selection
Original abstract (expand)
This note describes a Bayesian model selection or optimization procedure for post hoc inferences about reduced versions of a full model. The scheme provides the evidence (marginal likelihood) for any reduced model as a function of the posterior density over the parameters of the full model. It rests upon specifying models through priors on their parameters, under the assumption that the likelihood remains the same for all models considered. This provides a quick and efficient scheme for scoring arbitrarily large numbers of models, after inverting a single (full) model. In turn, this enables the selection among discrete models that are distinguished by the presence or absence of free parameters, where free parameters are effectively removed from the model using very precise shrinkage priors. An alternative application of this post hoc model selection considers continuous model spaces, defined in terms of hyperparameters (sufficient statistics) of the prior density over model parameters. In this instance, the prior (model) can be optimized with respect to its evidence. The expressions for model evidence become remarkably simple under the Laplace (Gaussian) approximation to the posterior density. Special cases of this scheme include Savage-Dickey density ratio tests for reduced models and automatic relevance determination in model optimization. We illustrate the approach using general linear models and a more complicated nonlinear state-space model.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 73%
- The Interpolating Information Criterion for Overparameterized ModelsChris van der Heide, Robert Salomone, Fred Roosta, Michael W. Mahoney Liam Hodgkinson2026≈ 72%
- A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image SegmentationR.J.G. van Sloun, C.G.A. Viviers, P.H.N. de With, F. van der Sommen M.M.A. Valiuddin2026≈ 71%
- An Interpretable Probabilistic Approach for Demystifying Black-box Predictive ModelsCatarina Moreira and Yu-Liang Chou and Mythreyi Velmurugan and Chun Ouyang and Renuka Sindhgatta and Peter Bruza2020≈ 71%
- Practitioner Motives to Use Different Hyperparameter Optimization MethodsNiklas Hasebrook, Felix Morsbach, Marc-Andr\'e Z\"oller, J\"org Franke, Marius Lindauer, Frank Hutter, Ali Sunyaev Niclas Kannengie{\ss}er2026≈ 70%
- ≈ 70%
- Algorithm Selection with Probing Trajectories: Benchmarking the Choice of Classifier ModelQuentin Renau and Emma Hart2025≈ 70%
- Active Inference, Curiosity and Insightin corpus2017≈ 70%
- ≈ 69%
- Piecewise Deterministic Markov Processes for Bayesian Neural NetworksDimitri Perrin, Kerrie Mengersen, Clinton Fookes Ethan Goan2026≈ 69%
- Active inference and artificial reasoningLancelot Da Costa, Alexander Tschantz, Conor Heins, Christopher Buckley, Tim Verbelen, Thomas Parr Karl Friston2025≈ 69%
- Bayesian score calibration for approximate modelsDavid J Warne, David J Nott, Christopher Drovandi Joshua J Bon2026≈ 69%
- Bayesian model of individual learning to control a motor imagery BCIFabien Lotte (Potioc), J\'er\'emie Mattout C\^ome Annicchiarico2024≈ 69%
- A variational synthesis of evolutionary and developmental dynamicsDaniel Ari Friedman, Axel Constant, V. Bleu Knight, Thomas Parr, John O. Campbell Karl Friston2023≈ 68%
- ≈ 68%
- A Review of Feature Selection Methods Based on Mutual InformationPablo A. Est\'evez Jorge R. Vergara2015≈ 68%
- ≈ 67%
- Interpreting Language Model Parametersin corpus2026≈ 63%
- Model Alignment Searchin corpus2025≈ 63%
- ≈ 63%
- ≈ 62%
- ≈ 62%
- Collective intelligence: A unifying concept for integrating biology across scales and substratesin corpus2024≈ 61%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 61%
- ≈ 61%
- Active inference: demystified and comparedin corpus2021≈ 61%
- Psychological Steering of Large Language Modelsin corpus2026≈ 61%
- Anima Labs Phenomenology Pt1in corpus≈ 61%
Similar preprints — Semantic Scholar
Cited by (3)
- Active Inference, Curiosity and Insight
Minimizing expected variational free energy under a discrete-state Markov decision process generative model is sufficient to produce curiosity, epistemic learning, and insight without any additional m
- Active inference on discrete state-spaces: a synthesis
Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, plannin
- Life as we know it
Any ergodic random dynamical system possessing a Markov blanket will, almost surely, appear to engage in active inference and maintain autopoietic integrity—making biological self-organization not a r