Make a system larger, feed it more, and something changes in kind, not just degree. Beyond mere improvement, new abilities appear that smaller versions never showed. Quantity, pushed far enough, turns into quality.
The core idea
Scaling describes how capability grows with model size, data, and computation. Across a wide range these quantities predict performance with striking regularity. More reliably yields better, often along a smooth curve.
Why it holds
The mechanism is partly statistical. Larger models can represent more, and more data pins down more of the world, so the two together capture finer structure. Compute is the currency that buys both.
A deeper reading
The turn is that some abilities seem to switch on. A capacity absent at one scale appears at another, as if crossing a threshold. Smooth inputs can yield what look like discontinuous gains.
The tension within
Scaling is not limitless. Data and compute have costs and ceilings, and returns eventually diminish. The curve that has climbed so far need not climb forever.
Where it fails
The implication unsettles the field’s self-image. If capability tracks scale more than cleverness, progress may owe more to resources than to ideas. The lever is size as much as insight.
The larger point
Scaling turns quantity into capability, with size, data, and compute predicting performance and sometimes summoning new abilities. Its returns are real but bounded. It suggests progress leans heavily on resources, not ingenuity alone. The principle rewards the patience to state it precisely and the humility to mark its limits. Precision reveals what it truly claims; humility reveals where it quietly fails. Between these two disciplines lies genuine understanding, which is never the possession of a conclusion but the grasp of why the conclusion holds and exactly how far it reaches.