A search for the lowest point can end in a valley that is low but not lowest. Trapped in a local minimum, a method may miss a deeper one elsewhere. Whether this matters depends on the shape of the terrain.
The core idea
A local minimum is a point lower than its surroundings but not necessarily the lowest overall, which is the global minimum. Descent methods can settle in the former, missing the latter. Getting stuck is the classic fear.
Why it holds
The mechanism of the trap is locality. A method that only moves downhill cannot climb out of a basin, even a shallow one, so it halts where no local step improves. The horizon of a local rule is limited.
A deeper reading
The turn is that in high dimensions the fear is often overblown. Most stalling points are saddles that descent can escape, and the many local minima that exist tend to be nearly as good as the best. Bad traps are rarer than intuition suggests.
The tension within
The quality of minima matters more than their number. If most low points yield similar performance, finding the exact global minimum is unnecessary. Good enough is often genuinely good.
Where it fails
The implication reassures practice. The success of training vast models despite a landscape full of minima reflects that its low regions are broadly excellent. The global optimum need not be reached.
The larger point
Local minima are low points that need not be the lowest, and local descent can settle in them. In high dimensions, however, most obstacles are escapable saddles and local minima are often nearly optimal. Reaching the exact global minimum usually proves unnecessary. Seen this way, the concept is less a fact to be filed than a lens through which other facts arrange themselves. Its worth lies not in any single application but in the pattern of thought it makes available. To hold it clearly is to see a whole family of problems as variations on one theme, and to recognize the same shape recurring where it was not expected.
