Scoring probabilities
Accuracy only checks whether the final label is right. Log loss, also called cross entropy, instead grades the probability the model assigned to the true outcome. It rewards confident correct predictions and severely punishes confident mistakes.
How it works
For each example, log loss takes the negative logarithm of the probability given to the true class.
- Predict 0.99 for the true class and the penalty is tiny.
- Predict 0.50 and the penalty is moderate.
- Predict 0.01 for the true class and the penalty explodes toward infinity.
You then average these penalties over all examples. Lower log loss is better.
Why it matters
Because the logarithm blows up near zero, a single overconfident wrong prediction can dominate the score. This pressures the model toward honest, calibrated probabilities rather than reckless certainty.
Key idea
Log loss grades the probability assigned to the truth and grows without bound as the model becomes confidently wrong, pushing it toward honest probabilities instead of bold guesses.