Scoring probabilities
Log loss, also called cross entropy, scores predicted probabilities rather than hard labels. For each example it takes the negative log of the probability the model assigned to the true class, then averages over the dataset. Lower is better.
Why confidence matters
The log term makes the penalty grow without bound as a confident prediction is proven wrong. Predicting point nine nine for a class that turns out false costs far more than predicting point six. This pushes models to report honest probabilities instead of bluffing.
- Correct and confident earns a tiny penalty near zero.
- Wrong and confident earns a huge penalty.
- Hedged near the base rate earns a moderate steady penalty.
Where it is used
Log loss is the loss many classifiers optimize during training and a natural evaluation metric when probabilities matter, such as risk scoring or ranking. Because it rewards good calibration, a model can have high accuracy yet poor log loss if its confidence levels are off.
Key idea
Log loss penalizes the negative log probability assigned to the true class, so confident wrong predictions are punished heavily and honest calibrated probabilities are rewarded.