Uncategorized
Transfer: Knowledge That Moves Between Tasks
What is learned in one place can pay off in another.
Uncategorized
Uncategorized
Distillation: Compressing One Mind Into a Smaller One
A large model's competence need not stay large.
Uncategorized
The Lottery Ticket Idea: Sparse Networks Hidden in Dense Ones
Inside a large trained network may hide a tiny one that could have learned the task alone.
Uncategorized
Double Descent: When Bigger Models Learn Better Again
Conventional wisdom says a model too large will overfit.
Uncategorized
No Free Lunch: The Impossibility of a Universal Learner
There is no algorithm that is best at everything.
Uncategorized
The Manifold Hypothesis: Why Data Lives on a Lower Dimension
High-dimensional data is rarely as vast as it looks.
Uncategorized
Regularization as a Statement of Belief
To regularize a model is to lean on it, gently, in a chosen direction.
Uncategorized
The Bias-Variance Tension at the Heart of Fitting
Every model that fits data faces a pull in two directions.
Uncategorized
Inductive Bias: The Prior That Makes Learning Possible
No learner learns from data alone.