What it is
AdaBoost, short for adaptive boosting, builds a strong classifier from a sequence of weak learners, often single split decision stumps. Each weak learner barely beats random guessing, but together they are powerful.
How it adapts
The clever part is reweighting.
- Every training example starts with an equal weight.
- After each weak learner, misclassified points get more weight.
- The next learner is forced to focus on the hard cases the others missed.
Combining the learners
- Each weak learner gets a vote weight based on its accuracy.
- More accurate learners earn a louder vote.
- The final prediction is a weighted majority of all the stumps.
Strengths and risks
- It needs few parameters and often works well out of the box.
- It is sensitive to noisy labels and outliers, since they keep getting upweighted.
- It assumes each weak learner is at least slightly better than chance.
Key idea
AdaBoost reweights misclassified examples each round and combines weak learners by accuracy weighted voting into a strong classifier.