What it is
An autoencoder is a network trained to copy its input to its output through a narrow middle. By forcing the data through a small bottleneck, it learns a compact representation called the latent code.
Two halves
An autoencoder is built from two stacked networks.
- The encoder maps the input down to a low dimensional latent code
- The decoder maps the latent code back up to a reconstruction of the input
Training minimizes a reconstruction loss, the difference between the input and the rebuilt output. No labels are needed, so this is self supervised learning.
What it learns and uses
The bottleneck is the whole point. It cannot store everything, so it must keep only the most useful structure.
- The latent code works as a learned dimensionality reduction
- A denoising autoencoder is fed corrupted input and must rebuild the clean version, which yields robust features
- Trained autoencoders also power anomaly detection, since rare inputs reconstruct poorly and produce high error
Key idea
An autoencoder squeezes data through a bottleneck and rebuilds it, learning a compact representation without labels.