Order can matter
Most training shuffles data randomly, but curriculum learning deliberately orders examples, typically from easy to hard, mirroring how people often learn. The idea is that mastering simple cases first gives a foundation that makes harder cases easier to learn.
How a curriculum is built
- A difficulty measure ranks examples, by length, rarity, or model loss.
- Training starts on the easier subset.
- Harder examples are gradually introduced as the model improves.
The schedule controls how quickly difficulty rises.
The progression
Benefits and cautions
Curricula can speed up convergence and sometimes improve final quality, especially on hard or noisy tasks. But they add complexity and depend on a sensible difficulty measure; a poor ordering can hurt. Anti curricula, hard to easy, occasionally help too, so the right schedule is empirical rather than guaranteed.
Key idea
Curriculum learning orders training from easy to hard using a difficulty measure, which can speed convergence and improve learning but depends on a sensible schedule.