To learn is to go beyond what one has seen. A system that merely records its examples has memorized; a system that performs well on the unseen has generalized. The distinction is the whole of learning, and it is more subtle than it appears.
The principle
Generalization is the capacity to extend from the observed to the unobserved. It presumes that the world has regularity a learner can capture. Without that assumption no inference from the past to the future could be justified.
The mechanism
The mechanism is compression. A learner that captures the rule rather than the instances describes its data more briefly and applies more widely. Brevity and breadth turn out to be two faces of the same virtue.
An unexpected turn
Yet the same flexibility that permits generalization permits memorization. A model powerful enough to learn the pattern is powerful enough to learn the noise. The art lies in preferring the former without forbidding the capacity for the latter.
The hidden cost
Generalization fails when the future stops resembling the past. A learner that has captured yesterday’s regularity is defenseless against a genuine change of regime. No amount of training data can insure against a world that turns.
The limit
The implication is that intelligence is not accumulation but abstraction. To know is to hold a compressed account of experience that reaches past it. The measure of understanding is performance on what was never shown.
The larger point
Generalization, not memory, is the aim of learning, and it rests on an unprovable faith that the world is regular. Its engine is compression and its enemy is the noise it is powerful enough to absorb. Understanding is what remains when the particulars are forgotten. What makes the idea durable is not that it settles a question but that it reframes many. It teaches where to look and what to discount, which is often more valuable than any particular answer it yields. Understood in this spirit, it becomes a habit of attention rather than a doctrine, and habits of attention are what distinguish deep comprehension from mere knowledge.
