paper:doi-10-1103-physrev-106-620Information Theory and Statistical Mechanics
Original abstract (expand)
Information theory provides a constructive criterion for setting up probability distributions on the basis of partial knowledge, and leads to a type of statistical inference which is called the maximum-entropy estimate. It is the least biased estimate possible on the given information; i.e., it is maximally noncommittal with regard to missing information. If one considers statistical mechanics as a form of statistical inference rather than as a physical theory, it is found that the usual computational rules, starting with the determination of the partition function, are an immediate consequence of the maximum-entropy principle. In the resulting "subjective statistical mechanics," the usual rules are thus justified independently of any physical argument, and in particular independently of experimental verification; whether or not the results agree with experiment, they still represent the best estimates that could have been made on the basis of the information available.It is concluded that statistical mechanics need not be regarded as a physical theory dependent for its validity on the truth of additional assumptions not contained in the laws of mechanics (such as ergodicity, metric transitivity, equal a priori probabilities, etc.). Furthermore, it is possible to maintain a sharp distinction between its physical and statistical aspects. The former consists only of the correct enumeration of the states of a system and their properties; the latter is a straightforward example of statistical inference.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 75%
- The Poincar\'e-Boltzmann Machine: from Statistical Physics to Machine Learning and backPierre Baudot2019≈ 74%
- Measuring the integrated information of a quantum mechanismRobert Prentner, Ian Durham Larissa Albantakis2023≈ 74%
- Shannon information and integrated information: message and meaningAlireza Zaeemzadeh and Giulio Tononi2024≈ 74%
- ≈ 74%
- Measuring Integrated Information: Comparison of Candidate Measures in Theory and SimulationAnil K. Seth and Adam B. Barrett Pedro A.M. Mediano2019≈ 74%
- ≈ 74%
- Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy PrincipleAlexander Kearney2026≈ 73%
- ≈ 73%
- Information, Processes and Gamesin corpus≈ 72%
- Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future DirectionLin Wang2025≈ 72%
- Some interesting observations on the free energy principleLancelot Da Costa and Thomas Parr Karl Friston2021≈ 72%
- ≈ 72%
- ≈ 72%
- Probabilistic Principles for Biophysics and Neuroscience: Entropy Production, Bayesian Mechanics & the Free-Energy PrincipleLancelot Da Costa2024≈ 71%
- The free energy principle made simpler but not too simpleLancelot Da Costa, Noor Sajid, Conor Heins, Kai Ueltzh\"offer, Grigorios A. Pavliotis and Thomas Parr Karl Friston2023≈ 71%
- ≈ 68%
- ≈ 66%
- ≈ 66%
- Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiesin corpus2023≈ 64%
- ≈ 63%
- Active Inference, Curiosity and Insightin corpus2017≈ 63%
- ≈ 63%
- Learning without neurons in physical systemsin corpus2022≈ 63%
- ≈ 62%
- Cognitive glues are shared models of relative scarcities: the economics of collective intelligencein corpus2026≈ 62%
- Active Inference: A Process Theoryin corpus2017≈ 62%
- Life as we know itin corpus2013≈ 62%
- Active inference: demystified and comparedin corpus2021≈ 61%
- Collective intelligence: A unifying concept for integrating biology across scales and substratesin corpus2024≈ 61%
Similar preprints — Semantic Scholar
Cited by (4)
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Active Inference: A Process Theory
A single variational principle—minimizing variational free energy via gradient descent on a Markov decision process (MDP) generative model—is sufficient to derive neuronal dynamics that reproduce, wit
- Active inference on discrete state-spaces: a synthesis
Active inference on discrete state-spaces, formalized as partially observable Markov decision processes (POMDPs) with likelihood matrix A, transition matrix B, and prior D, unifies perception, plannin
- Life as we know it
Any ergodic random dynamical system possessing a Markov blanket will, almost surely, appear to engage in active inference and maintain autopoietic integrity—making biological self-organization not a r