thinker:jing-huangJing Huang
Authored papers (3)
Larger models learn tasks smaller models never learn, and the cause is data-induced competition for neurons, not expressivity: small models spend capacity on frequent or simple tasks, and their updates overwrite rare-task features before the next rare observation arrives. The argument starts from power-law scaling. With a compute-optimal exponent γ=0.34 and an asymptotic parameter exponent α≈0.46, α>γ implies that a smaller model's infinite-data loss still exceeds a larger model's finite-compute loss. The paper formalizes this as 'learnable via data scaling' versus 'learnable via model scaling' (Definitions 1–2). In a synthetic mixture of K=32 linear-regression tasks, Theorem 3 shows a width-N encoder keeps the N features with the highest utility π_kλ_kj. Theorem 4 and Proposition 6 show that once common-task residual falls, rare-task features stop being overwritten. A matched-frequency injection protocol isolates retention from frequency. The same pattern appears in OLMo models (4M–4B parameters) pretrained on Dolma v1.7 with injected comparison and modular-addition tasks at frequencies from 7.8×10⁻³ to 2.4×10⁻⁸. Only larger models learn the rare tasks. They carry more task features, such as a 1-D global-order subspace with 96% DAS interchange success. They also show near-orthogonal non-task gradients (7.58×10⁻⁵ at 1B versus 0.10 at 20M). The implication is that scaling is partly a data-mixture problem: raising a target task's frequency may substitute for adding parameters.
pyvene is an open-source Python library that unifies intervention-based research on PyTorch neural models by treating the intervention itself—rather than model surgery code—as the primitive abstraction, expressed in a serializable dict-based configuration that can be shared via HuggingFace. Prior libraries (BauKit, TransformerLens, nnsight, graphpatch, Transformer Debugger) either lack extensibility to recurrent and convolutional architectures or require sophisticated custom code for multi-source, cross-forward-pass interventions; pyvene resolves both limitations with Getter/Setter hooks that track state variables enabling intervention at arbitrary time steps in GRU and other recurrent models. The library ships with trainable intervention types including RotatedSpaceIntervention (Distributed Alignment Search), LowRankRotatedSpaceIntervention, and BoundlessRotatedSpaceIntervention, and reproduces Meng et al.'s factual-association localization result in GPT2-XL in approximately 20 lines of code. A second case study on Pythia-6.9B demonstrates that a 1D DAS intervention finds sparse, causally localized gender representations across layers, whereas a linear probe achieves near-100% classification accuracy almost everywhere—implying that high probe accuracy is insufficient evidence of causal relevance, and that trainable interventions provide a strictly more diagnostic test of whether a representation is mechanistically load-bearing for a behavior.
More papers — OpenAlex / S2
Affiliations (1)
- Stanford University(institute)
Co-authors (12)
- Christopher Potts18 shared
- Atticus Geiger12 shared
- Aryaman Arora9 shared
- Christopher D. Manning9 shared
- Daniel Wurgaft9 shared
- David Alvarez-Melis9 shared
- Ekdeep Singh Lubana9 shared
- Laura Ruis9 shared
- Naomi Saphra9 shared
- Noah D. Goodman9 shared
- Rachit Bansal9 shared
- Zheng Wang9 shared
Their work is cited by (5)
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention4× refs
- CausalGym: Benchmarking causal interpretability methods on linguistic tasks3× refs
- Addressing divergent representations from causal interventions on neural networks3× refs
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?3× refs
- From Memorization to Reasoning in the Spectrum of Loss Curvature1× refs
Other inbound relations (2)
Recent mentions (3)
- papers-typedjack-2025-memorization-reasoning.md
- papers-typedhuang-2026-larger-learn.md
- papers-typedwu-2024-pyvene-library.md