Bayesian Inference Basics
Bayesian inference treats unknown quantities as having probability distributions that we update as evidence arrives. It is a principled way to combine prior knowledge with data.
The three pieces
Bayes rule combines three ingredients into an updated belief.
- The prior is what we believe about a quantity before seeing data.
- The likelihood says how probable the observed data is for each possible value.
- The posterior is the updated belief after combining prior and likelihood.
In words, posterior is proportional to likelihood times prior.
What the posterior gives
Instead of a single best estimate, Bayesian inference returns a full distribution. That distribution carries uncertainty, so we can report credible intervals and propagate doubt into later decisions rather than pretending we know one exact value.
Priors as a feature
The prior can encode genuine knowledge or gentle regularization. With little data the prior dominates and keeps estimates sensible. As data accumulates the likelihood takes over and the posterior concentrates, so the prior matters less.
The computational hurdle
The hard part is the normalizing constant in the denominator. For simple models it has a closed form, but for complex ones we approximate the posterior with sampling or variational methods.
Key idea
Bayesian inference updates a prior with the likelihood of data to produce a posterior distribution that carries uncertainty.