What it is
Despite the name, logistic regression is a classification model. It is the workhorse linear method for predicting the probability that an example belongs to a class, and it remains a strong, interpretable baseline.
From scores to probabilities
Logistic regression first computes a weighted sum of the features, a linear score, just like linear regression. It then passes that score through the sigmoid function, which squashes any real number into the range zero to one:
- Large positive scores map near one
- Large negative scores map near zero
- A score of zero maps to one half, the decision boundary
The model is trained by minimizing cross entropy, which rewards confident correct probabilities and punishes confident mistakes.
Why it endures
- The weights are interpretable, showing each feature direction and strength
- It outputs genuine probabilities, useful for ranking and thresholds
- It extends to many classes with the softmax, and pairs naturally with L1 or L2 regularization
Its limit is that the decision boundary is linear, so it underfits when classes are not linearly separable unless you add engineered features.
Key idea
Logistic regression maps a linear score through the sigmoid to produce class probabilities, trained with cross entropy and prized for interpretability.