The Bayes Theorem
Bayes theorem is the rule for updating a belief when new evidence arrives. It connects what you believed before with what the data now tells you.
The four pieces
- The prior is your belief about a hypothesis before seeing evidence.
- The likelihood is the chance of the evidence if the hypothesis were true.
- The posterior is your updated belief after the evidence.
- The evidence term normalizes so probabilities sum to one.
The theorem says the posterior is proportional to the prior times the likelihood.
The base rate trap
The most famous lesson involves rare conditions. Suppose a disease affects one person in a thousand and a test is 99 percent accurate. A positive result still leaves the person more likely healthy than sick, because the huge healthy population produces many false positives that swamp the few true ones. Ignoring the low prior, the base rate, leads to wrong conclusions.
Why machine learning cares
Bayesian thinking underpins spam filters, naive Bayes classifiers, and probabilistic models that quantify uncertainty rather than output a single guess. It also formalizes how regularization can be seen as a prior on parameters.
Key idea
Bayes theorem updates a prior belief by the likelihood of new evidence into a posterior, and ignoring low base rates produces badly wrong conclusions.