Conventional wisdom says a model too large will overfit. Yet past a certain size the rule inverts, and enormous models generalize better than moderate ones. The curve of error, plotted against capacity, descends twice.
The core idea
Double descent describes error that falls, rises, and falls again as model capacity grows. The first descent is classical improvement; the peak is the point of just-barely fitting the data. Beyond it, adding capacity paradoxically helps.
Why it holds
The mechanism concerns how a model interpolates. When capacity vastly exceeds the data, many perfect fits exist, and the training process tends to select smooth ones. Abundance of parameters permits a gentler solution.
A deeper reading
The turn overthrows a settled intuition. The old fear that capacity must be limited to avoid overfitting is, in this regime, simply wrong. Scale becomes an ally rather than a hazard.
The tension within
The picture is not a license for carelessness. The dangerous peak is real, and models of intermediate size can be the worst of all. Safety lies at neither the small nor the middle but the large.
Where it fails
The implication is that our theories trailed our practice. The success of vast models demanded an explanation the classical account could not give. Understanding followed capability rather than preceding it.
The larger point
Double descent shows error falling twice as capacity grows, redeeming the very largeness the old theory feared. Overparameterization can smooth rather than spoil. The relationship between size and generalization is deeper than a single trade-off. The principle rewards the patience to state it precisely and the humility to mark its limits. Precision reveals what it truly claims; humility reveals where it quietly fails. Between these two disciplines lies genuine understanding, which is never the possession of a conclusion but the grasp of why the conclusion holds and exactly how far it reaches.