Gaussian Mixture Models
A Gaussian mixture model, or GMM, describes data as a blend of several bell shaped clusters. Each cluster is a Gaussian with its own center, spread, and weight.
Soft clustering
Unlike k means, which hard assigns each point to one cluster, a GMM gives soft assignments. Every point holds a probability of belonging to each Gaussian. A point between two clusters can be sixty percent one and forty percent the other.
What the model holds
- A mixing weight for each component saying how common it is.
- A mean vector locating each component's center.
- A covariance describing each component's shape and orientation.
Covariance lets clusters be stretched ellipses, not just circles, so a GMM fits elongated groups that k means handles poorly.
Fitting with EM
GMMs are trained with expectation maximization. The E step computes each point's membership probabilities, and the M step updates the weights, means, and covariances. The process repeats until the likelihood settles.
Choosing components
The number of components is a choice. Too few underfit and too many overfit, so criteria that penalize complexity help pick a sensible count.
Key idea
A Gaussian mixture models data as weighted elliptical Gaussians with soft memberships fit by EM.