Handling Imbalanced Classes
In fraud, disease, and defect detection the interesting class is rare. When ninety nine percent of examples are negative, a model can score high accuracy by always predicting negative while catching nothing useful.
Why accuracy lies
With heavy imbalance, accuracy is misleading. Better metrics focus on the rare class:
- Precision asks how many flagged items were truly positive.
- Recall asks how many true positives were caught.
- Area under the precision recall curve summarizes the tradeoff.
Techniques to cope
- Resampling either oversamples the minority class or undersamples the majority to balance the training set.
- Class weights tell the loss function to penalize minority mistakes more heavily.
- Threshold tuning moves the decision cutoff to trade precision against recall for the business need.
A caution
Balancing the training data changes the base rate the model assumes, so probabilities may need calibration afterward. The right choice depends on the cost of a missed positive versus a false alarm, which is a business decision, not a purely technical one.
Key idea
Under imbalance, accuracy misleads, so use recall and precision and apply resampling, class weights, or threshold tuning to surface the rare class.