Uncategorized
Distillation: Compressing One Mind Into a Smaller One
A large model's competence need not stay large.
Stories by Mahendra
Uncategorized
The Lottery Ticket Idea: Sparse Networks Hidden in Dense Ones
Inside a large trained network may hide a tiny one that could have learned the task alone.
Uncategorized
Double Descent: When Bigger Models Learn Better Again
Conventional wisdom says a model too large will overfit.
Uncategorized
No Free Lunch: The Impossibility of a Universal Learner
There is no algorithm that is best at everything.
Uncategorized
The Manifold Hypothesis: Why Data Lives on a Lower Dimension
High-dimensional data is rarely as vast as it looks.
Uncategorized
Regularization as a Statement of Belief
To regularize a model is to lean on it, gently, in a chosen direction.
Uncategorized
The Bias-Variance Tension at the Heart of Fitting
Every model that fits data faces a pull in two directions.
Uncategorized
Inductive Bias: The Prior That Makes Learning Possible
No learner learns from data alone.
Uncategorized
Generalization and the Difference Between Learning and Memorizing
To learn is to go beyond what one has seen.