Hidden Markov Models
A hidden Markov model, or HMM, describes a sequence where the true state is hidden and we only see noisy observations. It is a workhorse for speech, text, and biological sequences.
The two layers
An HMM has a chain of hidden states and a stream of observations.
- Hidden states follow the Markov property, meaning the next state depends only on the current state.
- Each hidden state emits an observation through an emission distribution.
- We never see the states directly, only the emissions.
The three ingredients
The model is defined by initial state probabilities, a transition matrix giving state to state probabilities, and emission probabilities linking states to observations.
The classic questions
HMMs answer three problems. Given a model and observations, how likely is the sequence, solved by the forward algorithm. What hidden path best explains the observations, solved by the Viterbi algorithm. And how do we learn the parameters from data, solved by an EM procedure.
Why the Markov assumption helps
Because the next state depends only on the present, computations factor into small steps, making otherwise huge sums over all possible state paths tractable.
Key idea
An HMM models hidden states that follow the Markov property and emit the noisy observations we actually see.