paper:doi-10-1162-neco-1995-7-5-889The Helmholtz Machine
Original abstract (expand)
Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- Can a Machine be Conscious? Towards Universal Criteria for Machine ConsciousnessCosmin Badea Nur Aizaan Anwar2024≈ 63%
- On the Implicit and on the Artificial - Morphogenesis and Emergent Aesthetics in Autonomous Collective SystemsVitorino Ramos2007≈ 61%
- From Masks to Worlds: A Hitchhiker's Guide to World ModelsYu Lei, Hecong Wu, Yuchen Zhu, Shufan Li, Yi Xin, Xiangtai Li, Molei Tao, Aditya Grover, Ming-Hsuan Yang Jinbin Bai2025≈ 61%
- ≈ 61%
- A Survey of Theory of Mind in Large Language Models: Evaluations, Representations, and Safety RisksHieu Minh "Jord" Nguyen2025≈ 61%
- ≈ 61%
- ≈ 61%
- The Poincar\'e-Boltzmann Machine: from Statistical Physics to Machine Learning and backPierre Baudot2019≈ 61%
- ≈ 61%
- Brief Architectural Survey of Biopotential Recording Front-Ends since the 1970sMinkyu Je Taeju Lee2023≈ 60%
- Low count of optically pumped magnetometers furnishes a reliable real-time access to sensorimotor rhythmDaria Medvedeva (1,2), Oleg Shevtsov (2,1) and Alexei Ossadtchi (1,2) ((1) Centre for Bioelectric Interfaces Higher School of Economics, Moscow, Russia, (2) LIFT, Life Improvement by Future Technologies Institute Moscow, Russia) Nikita Fedosov (1)2024≈ 60%
- ≈ 60%
- ≈ 60%
- ≈ 60%
- Auxilio and Beyond: Comparative Evaluation, Usability, and Design Guidelines for Head Movement-based Assistive Mouse ControllersMohammad Ishrak Abedin (2), Rizvi Ahmed (2), Saad Bin Ashraf (2), Hasan Mahmud (3), Md. Kamrul Hasan (3) ((1) iStudio Lab, School of Computing, Queen's University, Kingston, Ontario, Canada, (2) Network and Data Analysis Group (NDAG), Department of Computer Science and Engineering (CSE), Islamic University of Technology (IUT), Board Bazar, Gazipur, Bangladesh, (3) Systems and Software Lab (SSL), Department of Computer Science and Engineering (CSE), Islamic University of Technology (IUT), Board Bazar, Gazipur, Bangladesh.) Mohammad Ridwan Kabir (1)2026≈ 60%
- ≈ 60%
- ≈ 60%
- ≈ 59%
- Relating transformers to models and neural representations of the hippocampal formationin corpus2021≈ 59%
- ≈ 59%
- Collective intelligence: A unifying concept for integrating biology across scales and substratesin corpus2024≈ 59%
- ≈ 58%
- ≈ 58%
- ≈ 57%
- ≈ 57%
- ≈ 57%
- A Mathematical Framework for Transformer Circuitsin corpus2021≈ 57%
- The World Inside Neural Networksin corpus2026≈ 57%
- The Machine Consciousness Hypothesisin corpus≈ 57%
Similar preprints — Semantic Scholar
Cited by (3)
- A tale of two densities: active inference is enactive inference
Ramstead, Kirchhoff, and Friston argue that generative models in active inference under the free energy principle (FEP) are control systems—not structural representations—and that this distinction has
- Active inference on discrete state-spaces: a synthesis
Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, plannin
- Life as we know it
Any ergodic random dynamical system possessing a Markov blanket will, almost surely, appear to engage in active inference and maintain autopoietic integrity—making biological self-organization not a r