Mode Collapse In GANs
Mode collapse is a failure where a GAN generator produces only a narrow set of outputs, ignoring much of the real data variety.
What it looks like
- A face generator that outputs almost the same face every time.
- A digit generator that only ever makes a few digits and never the others.
- Samples look sharp individually but lack diversity across the dataset.
Why it happens
- The generator finds a single output that reliably fools the current discriminator and keeps producing it.
- The discriminator then learns to reject that output, so the generator hops to another single mode.
- This cat and mouse cycle can repeat without the generator ever covering the full distribution.
How to fight it
- Minibatch features let the discriminator look at a batch and penalize repetition.
- Unrolled or Wasserstein objectives smooth the training signal so the generator cannot exploit a brittle discriminator.
- Adding noise or feature matching encourages broader coverage.
Key idea
Mode collapse is when a GAN generator covers only a few modes of the data, and remedies like minibatch features and the Wasserstein objective push it toward fuller coverage.