From goal to task
A business goal like reduce churn is not an ML task. You must choose inputs, outputs, and a learning paradigm.
- Classification predict a discrete label, such as will churn or not
- Regression predict a number, such as days until churn
- Ranking order items, such as which users to contact first
Two layers of metrics
Separate what the model optimizes from what the business cares about.
- Model metrics precision, recall, AUC, RMSE
- Business metrics retained revenue, conversion rate, cost per action
A model can score high on AUC and still lose money if the threshold is wrong. Always connect a model metric to a business outcome.
Choosing wisely
- Use precision when false positives are expensive
- Use recall when missing positives is expensive
- Use F1 or PR AUC under heavy class imbalance, not plain accuracy
State your metric out loud and justify it against the cost of each error type. That single sentence shows senior judgment.
Key idea
Pick a metric that mirrors the real cost of being wrong, then tie it back to a business number you can defend.