Normalizing Flows
A normalizing flow builds a generator out of invertible transformations. It warps a simple distribution like a Gaussian into a complex data distribution while keeping exact likelihoods.
The core idea
- Start with a simple base distribution that is easy to sample.
- Pass samples through a chain of invertible, differentiable layers.
- The output follows a rich, complex distribution shaped by the layers.
Exact likelihood
- Because each layer is invertible, the change of variables formula gives the exact density of any data point.
- This requires the Jacobian determinant of each transform, so layers are designed to make that determinant cheap to compute.
- Unlike VAEs and GANs, flows give a tractable exact likelihood, which is useful for density estimation and anomaly detection.
Tradeoffs
- The invertibility constraint limits layer design, so flows can need many layers to match GAN or diffusion quality on images.
- They sample and evaluate likelihood in a single pass, which is fast at inference.
Key idea
Normalizing flows map simple noise to complex data through invertible layers, and the change of variables formula gives an exact likelihood that GANs and VAEs cannot provide.