The idea
An autoencoder is a neural network trained to copy its input to its output. That sounds pointless until you force the data through a narrow middle layer, the bottleneck, which compels the network to learn a compact, meaningful code.
Two halves
An autoencoder has two parts that train together:
- The encoder compresses the input into the low dimensional bottleneck representation
- The decoder reconstructs the original input from that code
- Training minimizes the reconstruction error, the gap between input and output
Because the bottleneck is too small to memorize everything, the network must capture only the most important structure, a learned nonlinear cousin of principal component analysis.
What they are good for
- Dimensionality reduction and feature learning without labels
- Anomaly detection, since unusual inputs reconstruct poorly and get high error
- Denoising, by training to rebuild a clean input from a corrupted one
A plain autoencoder learns an unstructured code, so generating new samples from it is unreliable, which motivates the variational version.
Key idea
An autoencoder learns a compact code by squeezing input through a bottleneck and reconstructing it, useful for compression, anomaly detection, and denoising.