Predicting which customers are about to leave is one of the most common uses of machine learning in business. It is also one of the most frequently wasted. Teams celebrate a model with excellent accuracy, the campaign is sent to a list chosen by habit, and a year later nobody can say whether the investment paid off.
Translate the model into money
Four numbers turn a churn model into a business case: how many customers would leave, the value of keeping one, the share of contacted churners the offer actually saves, and the cost of each contact. Rank the customers by the model's risk score, and for each possible campaign size compute the value saved minus the cost of contacting them.
Contacting everyone earns 90 thousand. Contacting the 46% of customers the model ranks as most at risk earns about 151 thousand, two thirds more, with less than half the contacts. The same 46% chosen at random would earn only about 41 thousand. Beyond the optimum each additional contact costs more than it saves, because the remaining customers were unlikely to leave anyway.
Three refinements that add value
- Value, not only risk: a customer with a 30% risk who is worth ten times more deserves a call before one with 60% risk and low value.
- Who can be persuaded: uplift models estimate who changes behaviour because of the offer, so you do not pay to keep customers who would have stayed, or provoke the ones who would not.
- Always keep a control group: a small random share of high-risk customers that is not contacted is the only reliable proof of what the campaign earned.
Machine learning creates value only when it changes a decision. In churn, the decision is who to contact and with what offer, and the right measure of success is the profit of that decision, verified against a control group.
Bartor S.
Founder
