paper
referenced-only
2019
paper:doi-10-1002-cpa-22008The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
BySong Mei·A. Montanari
Original abstract (expand)
Deep learning methods operate in regimes that defy the traditional statistical mindset. Neural network architectures often contain more parameters than training samples, and are so rich that they can interpolate the observed labels, even if the latter are replaced by pure noise. Despite their huge complexity, the same architectures achieve small generalization error on real data.
Similar preprints — Semantic Scholar
Cited by (1)
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
Larger models learn tasks smaller models never learn, and the cause is data-induced competition for neurons, not expressivity: small models spend capacity on frequent or simple tasks, and their update