paper:doi-10-1038-s41467-018-03101-6Input–output maps are strongly biased towards simple outputs
Original abstract (expand)
Many systems in nature can be described using discrete input-output maps. Without knowing details about a map, there may seem to be no a priori reason to expect that a randomly chosen input would be more likely to generate one output over another. Here, by extending fundamental results from algorithmic information theory, we show instead that for many real-world maps, the a priori probability P(x) that randomly sampled inputs generate a particular output x decays exponentially with the approximate Kolmogorov complexity [Formula: see text] of that output. These input-output maps are biased towards simplicity. We derive an upper bound P(x) ≲ [Formula: see text], which is tight for most inputs. The constants a and b, as well as many properties of P(x), can be predicted with minimal knowledge of the map. We explore this strong bias towards simple outputs in systems ranging from the folding of RNA secondary structures to systems of coupled ordinary differential equations to a stochastic financial trading model.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- An Evaluation on Large Language Model Outputs: Discourse and MemorizationXun Wang, Alex Sokolov, Qilong Gu and Si-Qing Chen Adrian de Wynter2026≈ 70%
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?in corpus2025≈ 70%
- ≈ 70%
- ≈ 69%
- Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic InterpretabilityTessa Han, Usha Bhalla, Himabindu Lakkaraju Shichang Zhang2025≈ 69%
- Analyze Feature Flow to Enhance Interpretation and Steering in Language ModelsNikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov Daniil Laptev2025≈ 69%
- ≈ 69%
- CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image EncodersL.B. Soros, Olaf Witkowski Kevin Frans2021≈ 69%
- ≈ 69%
- Mechanistic Indicators of Steering Effectiveness in Large Language ModelsHao Xue, Flora Salim Mehdi Jafari2026≈ 68%
- ≈ 68%
- Locating Demographic Bias at the Attention-Head Level in CLIP's Vision EncoderKittipat Phunjanna, Marcos Escudero Vi\~nolo, Catarina Barata, Jenny Benois-Pineau Alaa Yasser2026≈ 68%
- Improving Steering Vectors by Targeting Sparse Autoencoder FeaturesSviatoslav Chalnev and Matthew Siu and Arthur Conmy2024≈ 68%
- Recommender systems, stigmergy, and the tyranny of popularityPaul E. Smaldino Zackary Okun Dunivin2025≈ 68%
- Interpretability and Generalization Bounds for Learning Spatial PhysicsTheo Gutman-Solo, Shuai Jiang Alejandro Francisco Queiruga2026≈ 68%
- Visual Representations inside the Language ModelAmita Kamath, Madeleine Grunde-McLaughlin, Winson Han, Ranjay Krishna Benlin Liu2025≈ 68%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 66%
- Model Alignment Searchin corpus2025≈ 66%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 66%
- ≈ 66%
- The Platonic Representation Hypothesisin corpus2024≈ 66%
- Why Learning Requires Feelingin corpus2026≈ 65%
- ≈ 65%
- Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiesin corpus2023≈ 65%
- Active inference: demystified and comparedin corpus2021≈ 65%
- ≈ 65%
- Psychological Steering of Large Language Modelsin corpus2026≈ 64%
Similar preprints — Semantic Scholar
Cited by (2)
- The Platonic Representation Hypothesis
Neural networks trained on different data modalities, architectures, and objectives are converging toward a shared statistical model of reality — what the paper terms the "platonic representation" — f
- The computational boundary of a 'self': developmental bioelectricity drives multicellularity and scale-free cognition
Scale-Free Cognition, the framework introduced here, proposes that any coherent Self is demarcated by a 'cognitive light cone'—a spatio-temporal boundary of events a system can measure, model, and att