paper:doi-10-48550-arxiv-2609-04963Fractal basins trap latent reasoning
Methods (13)
- Basin EntropyMetric distinguishing fractal from smooth/random basins, computed by partitioning slices into boxes and taking mean Shannon entropy of settling times.
- Boundary Basin EntropyVariant of basin entropy averaged only over boxes straddling multiple basin values.
- Convergence-Time Basin Mapping ProbeThe paper's new probe: continuously varying a model's initial latent state along 2D slices and labeling outcomes by convergence time.
- Encoder-Only Looped Transformer for Integer Linear SystemsMiniaturized model the authors train themselves to directly observe the training-time bifurcation into fractal basins.
- Equilibrium Reasoners (EqR)One of four reasoning architectures probed; iterates paired latents, used on Sudoku-Extreme and Maze-Unique.
- Fast Lyapunov Indicator (λF)Metric directly localizing saddle-like boundaries between solution routes by measuring maximal local trajectory separation.
- Finite-Time Lyapunov Exponents (FTLE)Spectrum quantifying amplification/suppression of perturbations along independent latent directions, used to detect transient chaos onset during training.
- Fixed-Point Reasoning Models (FPRM)Architecture iterating a single latent to a fixed point via residual threshold; used on Sudoku and mazes.
- Hierarchical Reasoning Model (HRM)Cited small (7M param) recurrent model that outperforms >10B-parameter LLMs on ARC-AGI, motivating the paper's focus on reasoning dynamics.
- Parcae140M-parameter looped language model, fine-tuned on Countdown arithmetic to probe basin fractality on mathematical logic.
- Principal components analysis (PCA)Statistical method used to analyze neural activity data.
- Tiny Recursive Model (TRM)Architecture used to probe ARC-AGI-1 visual puzzle basins.
- Uncertainty Exponent (α)Metric estimating basin-boundary fractal dimension from how the fraction of differing-outcome pixel pairs scales with separation.
Frameworks (1)
- "Plinko" model of high-dimensional dynamicsThe paper's illustrative model: closely-spaced trajectories scatter unpredictably off weakly-unstable saddle sets en route to equilibrium.
Datasets (5)
- ARC-AGI-1 (Abstract Reasoning Corpus)Visual-puzzle benchmark used with TRM to probe fractal basins, and cited as evidence for reasoning models' efficiency.
- Countdown (arithmetic task)Mathematical logic task used to fine-tune Parcae and probe basin fractality.
- Maze-Hard30×30 maze test split used with FPRM, difficulty annotated by shortest-path length.
- Maze-UniqueMaze test split used with EqR checkpoints.
- Sudoku-ExtremeSudoku benchmark test set with annotated backtracking-based difficulty, used to probe basins under varying difficulty.
Findings (17)
- Fractal basins and their scaling with difficulty replicate on alternative architecture-task pairings (EqR-Maze, encoder-only looped transformer on Countdown)
Replication check ruling out that fractal basins arise only from one model-task pairing.
- Basin entropy strongly correlates with number of reasoning iterations required to converge, across Sudoku, mazes, visual puzzles, and mathematical logic
Core empirical result establishing fractality scales with task difficulty across four tasks/architectures.
- A bifurcation in the miniaturized looped transformer occurs at ~150,000 training steps, when accurate integer-linear-system solving first emerges
Pinpoints the training-time transition where fractal basins emerge.
- Faster-converging trajectories take direct PCA-space paths to the true solution; slower trajectories take indirect routes visiting saddle points
PCA-space visualization linking route geometry to convergence speed.
- Fast Lyapunov indicator λF strongly correlates with the number of distinct solutions a reasoning trajectory passes through before converging
Links saddle-crossing intensity to the number of candidate-answer switches during reasoning.
- Only the multistep 'core' variables (not directly-substitutable variables) produce positive FTLE in the trained looped transformer
Localizes transient chaos to the sub-algorithm requiring multi-step Gaussian elimination.
- Decoding the latent state near a saddle point produces a nearly-correct solution attempt (repeated Sudoku digit / maze dead end)
Directly identifies saddle points with near-miss solution attempts, the mechanistic core of the paper's account.
- Parcae-Countdown uncertainty exponent across scales: α=0.912±8e-3 (1×), 0.846±6e-3 (10×), 0.861±3e-2 (100×), 0.887±2e-2 (1000×)
Third architecture/task pair confirming near-scale-invariant fractal basin structure.
- Adversarially-chosen prompts induce denial-of-service attacks consuming 10× more compute than benign prompts
Background finding illustrating the severity of uncontrolled reasoning slowdowns.
- EqR-Sudoku uncertainty exponent decreases with zoom: α=0.249±6e-3 (1×), 0.243±9e-3 (10×), 0.216±4e-3 (100×)
Evidence that EqR-Sudoku basins are 'slim fractals' — scale-dependent, decreasing fractal complexity with resolution.
Claims (14)
- Hardness-induced fractal basins are ubiquitous in modern reasoning models, emerging even without hand-designed task-specific solvers, across highly distinct architectures and tasks
The paper's generalizing claim distinguishing its contribution from prior narrow demonstrations (e.g., Ercsey-Ravasz & Toroczkai's SAT solver).
- Reasoning models' fractal basins are smooth and naturalistic, resembling optical caustics or turbulent transient lifetimes, despite the discrete puzzles being solved
Visual/qualitative characterization of the basin geometry linking it to classical physical fractal phenomena.
- Slim fractals (scale-dependent fractal dimension) arise in physical systems with dissipation, where the saddle-like set decays over time
Explains why EqR-Sudoku basins show decreasing fractal dimension with zoom.
- Fractal basins form because reasoning trajectories become trapped near weakly-unstable saddle points for extended durations
The paper's core mechanistic claim connecting saddle dynamics to basin fractality.
- Fractal basins and transient chaos are the generic manner in which computational complexity manifests in analogue systems
Generalizing interpretation connecting this paper's AI findings to physical analogue computation broadly.
- Constraint-solving problems are hardest near the constrainedness transition, where few distinct correct routes remain
Interpretive claim linking classical constraint-satisfaction hardness theory to the paper's saddle-based account.
- Saddle points in reasoning latent space correspond to nearly-correct attempted solutions of the underlying problem
Interprets the decoded latent states near saddles (Fig.3D) as near-miss answers.
- Modern reasoning models can be placed within the longstanding tradition framing computation as a physical process
Closing claim connecting to Moore/Shaw/Crutchfield-style physical computation theory.
- Saddle points shape the reasoning landscape by encoding the solution structure of the underlying problem
Interprets the λF–solution-switch-frequency correlation as evidence that saddles are algorithmically meaningful.
- Reasoning models' capability is tied to their ability to escape saddle points near incorrect solutions
Practical interpretive upshot connecting dynamics to model competence.
Hypotheses (1)
- Most reasoning models are expected to exhibit some form of doubly transient chaos, because properties like convergence-to-fixed-answers act as a dissipation-like mechanism
Forward-looking predictive claim about reasoning models generally, based on the analogy to damped physical systems.
Original abstract (expand)
Reasoning allows artificial intelligence models to revisit and correct their mistakes, enabling recent frontier advances in mathematical theorem solving, software engineering, and autonomous task planning. Reasoning models are widely observed to reason for longer on harder tasks, but the general mechanism responsible for these slowdowns is unknown. Here, we show that reasoning models exhibit transient chaos, a physical consequence of the computational complexity of difficult tasks. As a consequence, we show that diverse leading reasoning models are dynamical systems with fractal basins, with fractality increasing with task difficulty across diverse tasks like Sudoku and maze solving, visual puzzles, and mathematical logic. We show that transient chaos emerges due to reasoning becoming trapped for extended durations near saddle points, which we show correspond to nearly-correct attempted solutions of the underlying problem. Our results show that reasoning slowdowns are an inevitable consequence of problem hardness in modern artificial intelligence models, and establish reasoning traces as a rich new class of dynamical system.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMsLiner Yang, Mengyan Wang, Luming Lu, Weihua An, Erhong Yang Jiyuan An2026≈ 69%
- LoopBench: Discovering Emergent Symmetry Breaking Strategies with LLM SwarmsYashar Talebirad, Csongor Szepesv\'ari, Vishwajeet Ohal, Eden Redman Ali Parsaee2025≈ 69%
- ≈ 69%
- CausalARC: Abstract Reasoning with Causal World ModelsJohn Kalantari, Kia Khezeli Jacqueline Maasch2026≈ 69%
- Discrete Latent Structure in Neural NetworksCaio F. Corro, Nikita Nangia, Tsvetomila Mihaylova, Andr\'e F. T. Martins Vlad Niculae2026≈ 68%
- Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language ModelWei He, Xuhan Zhu, Chunpeng Zhou, Qizhen Li, Song Yan, Yufei Zheng, Chengjun Yu, Fan Lu, Wei Zhai, Yang Cao, Pengfei Yu, Zheng-Jun Zha Chenfeng Wang2026≈ 68%
- Latent Concept Disentanglement in Transformer-based Language ModelsBhavya Vasudeva, Vatsal Sharan, Cyrus Rashtchian, Prabhakar Raghavan, Rina Panigrahy Guan Zhe Hong2025≈ 68%
- ≈ 68%
- Reasoning emerges from constrained inference manifolds in large language modelsFei Luo, Linfeng Zhang, Chuangxin Zhao, Mingxuan Wang, Yinan Wu, Zhe Qian, Yang Lu, Long Chen, Zhao Cao, Xiaoshuai Hao, Ji-Rong Wen, Jungong Han Yanbiao Ma2026≈ 68%
- Reasoning with Language Model is Planning with World ModelYi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, Zhiting Hu Shibo Hao2023≈ 68%
- Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent SpaceTim Seyde, Igor Gilitschenski, Lucas Liebenwein, Ryan Sander, Sertac Karaman, Daniela Rus Wilko Schwarting2021≈ 68%
- Modeling Human Behavior Part I -- Learning and Belief ApproachesAndrew Fuchs and Andrea Passarella and Marco Conti2022≈ 68%
- ≈ 68%
- From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous SystemsSiyu Chen, Fengzhuo Zhang, Zhuoran Yang Jianliang He2024≈ 68%
- ≈ 68%
- ≈ 67%
- ≈ 66%
- ≈ 66%
- When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Modelsin corpus2025≈ 65%
- ≈ 65%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 65%
- ≈ 65%
- Learning without neurons in physical systemsin corpus2022≈ 65%
- ≈ 65%
- Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationsin corpus2023≈ 64%
- ≈ 64%
- ≈ 64%
- ≈ 64%
- ≈ 64%
- ≈ 64%
+8 more