The Diffusion Reverse Denoising
If the forward process destroys data, the reverse process rebuilds it. A neural network learns to denoise step by step, starting from pure noise and ending at a clean sample.
The learned denoiser
- A network takes a noisy sample and the current timestep and predicts the noise that was added.
- Subtracting the predicted noise moves the sample one step closer to clean data.
- Repeating this from pure noise down to step zero produces a brand new sample.
How it trains
- Take a clean example and add noise at a random timestep using the forward process.
- Ask the network to predict that noise.
- The loss is simply the squared error between true and predicted noise, which is stable and easy to optimize.
Why it generates well
- Each step makes only a small change, so errors do not compound the way they do in one shot generation.
- The same network handles every timestep, conditioned on which step it is on.
- Sampling can trade speed for quality by using more or fewer reverse steps.
Key idea
The reverse diffusion process trains a network to predict the noise in a corrupted sample, then denoises step by step from pure noise to a clean sample, with small steps keeping errors from compounding.