in marketing campaigns is very difficult. Much of it comes down to trial and error, although we know that targeted strategies can work better. We just don't know how to get there. The process usually involves launching a campaign, viewing it, reading it, making changes, and then trying again. This trial and error approach has real power. It encourages movement over disability. It allows teams to learn quickly, especially in fast-changing markets. In early stage growth or limited data areas, it is often the only viable option.
I want to present a different approach. One is, without a doubt, the most difficult, advanced, and complex, but also dynamic and remarkable. This is the path that takes companies to the next level of data maturity. Let me introduce you to expected value modeling.
Before we begin, I want to introduce that this approach takes up entire chapters in some data science books. However, I aim to be as non-technical as possible. I will keep the ideas conceptual, while still providing a clear outline of how this can be accomplished. If you would like to read more, I will cite useful sources at the end.
Let's get started.
What is Expected Value Modeling?
Expected value is an important analytical framework that allows decision makers to consider trade-offs where there are unequal costs and benefits. Consider a scenario where a machine learning model helps diagnose a patient with cancer. Frames and models that include only simple accuracy (whether the prediction was right or wrong) do not account for the trade-off of prediction.
In this case, not all “wrong predictions” are the same. Not diagnosing a patient with cancer when they have it is more expensive than diagnosing someone with cancer when they actually have it. Both predictions were technically incorrect, but one cost lives, the other did not.
Thankfully, our marketing strategies are not life or death situations. But this principle works the same. The decision of who to target in a marketing campaign, and who not to target, can result in very different costs for a business.
Expected Value Modeling expands this horizon to describe possible outcomes, and allows us to estimate each cost or benefit. This framework relies heavily on the business knowledge of subject matter experts to determine the implications of each outcome. Our goal here is to understand how to design a strategy that statistically optimizes our goal. In the remainder of this article, we will focus on learning who to target in a marketing plan to maximize profits.
Start with the Prospective Purchase model
A Purchase Probability Model is a machine learning model that predicts the likelihood that a customer will purchase a product. Let's imagine that we are running an ad campaign for an e-commerce business. Each person who clicks on an ad creates a line of data. They see the campaign, browse your store, and ultimately decide whether or not to buy the product. During this process, a number of data points need to be collected. A machine learning model analyzes all historical data to identify patterns. It learns what factors influence a customer's likelihood of making a purchase. Then, those patterns are applied to new customers to predict whether they will buy the product.
This model itself is overpriced. It tells the business who the customers are most likely to buy the product and what aspects of the campaign influence the likelihood of purchase. We may use this information to plan our next ad campaign. This is what data-driven decision making looks like.
Using Expected Value Modeling
Moving forward, it is important to understand the concept of a confusion matrix. The confusion matrix is a n x n the table there n represents all possible outcomes. For simplicity, I'll stick to a 2 x 2 confusion matrix.
This matrix contains predicted results on one axis and actual results on the other. It gives us four cells, one for each possible outcome in the binary classification problem, as does our model of purchase probability (whether the customer buys the product or not). This gives rise to the following possibilities:
- True Positive: we predicted that the customer would buy, and indeed they did.
- False positives: we predicted that the customer would buy, but they didn't.
- False Negatives: we predicted that the customer would NOT buy, but they did.
- True Negative: we predicted that the customer would NOT buy, and in fact they didn't.
Here is an illustration:
To apply expected values to each outcome we need to have a deep understanding of the business. We need to know the following information:
- Profit per product sold.
- Cost per click.
- Probability of buying for each customer.
In the same example of our e-commerce store, let's consider the following values:
- Profit per product sold = $50
- Cost per click = $1
- Probability of purchase per customer = from our Prospective Purchase model
Knowing this information we can determine that the profit of the customer clicking on our ad campaign and buying the product (True Good) would be the profit of each product ($50) subtract cost per click ($1), which is equivalent $49. The cost of a customer who clicks on our campaign but does not buy (False Positive) is the cost of the click, therefore. $1. The result of misunderstanding is a customer who will not buy $0since no expenses were incurred and no money was earned. The result of not targeting the buyer $0 for the same reasons.
I want to appreciate the opportunity cost of not being referred to a potential buyer or the chances of someone making a purchase without being referred. These are very abstract and subjective, although not possible to measure. For simplicity, I will not consider them in this case.
This leaves us with the following confusion matrix:
Great, now we know the actual cost or profit of each result of our ad campaign. This allows us to understand the expected value of targeting a customer using the following equation (sorry for throwing the math at you):
Expected Profit = P(buy) × Profit if you buy + (1 — P(buy)) × Loss if you don't buy
When the expected value is equal to response opportunities (P(buy)) times i feedback value (Profit if you buy) as well as Probability of not responding (1 — P(purchase)) times i cost of non-response (Loss if no purchase).
If we want the expected value of targeting a customer to be positive, which means we are profitable, we can rearrange the equation to the following:
P(buy) × $49 + (1 — P(buy)) × (–$1) > 0
P(buy) > 0.02 (or 2%)
This means that, based on our probability of purchase model, we must target each customer with a probability of purchase greater than 2%.
You don't need to have a degree in mathematics or statistics to use this, but I wanted to show how we got there.
We have our answer: we need to target all customers whose purchase probability is greater than 2%. Now we can go back to our opportunity model to see which customer segments fit the criteria.
We found out who to target, tailored our campaign to their needs, and implemented an effective marketing campaign. We built our strategy with all the right foundations by making real data-driven decisions.
Taking it one step further with Profit Curves
We have built our framework and designed our marketing campaign in a way that optimizes our ROI. However, there are often additional constraints that limit our ability to implement a campaign, often related to how much budget is allocated and how many people can be targeted. In these cases, it is useful to know not only the correct decision, but also the expected value of the various possibilities. In those cases, we can embed the expected value calculation into our process of training the probability model.
Instead of selecting models based solely on technical performance, we can evaluate them based on expected profitability. Or use an integrated approach that balances predictive power and economic impact.
While building our model, we can calculate the expected profit for the entire range of people we can target, from targeting no one to everyone we know. As a result, we get the plot of the profit curve:
On the y-axis we have the expected profit of the marketing campaign based on how many people we targeted. On the x-axis we have the purchase limit. We are getting less and less with our campaign as we increase the margin. If we can go up to 100%, we will not target anyone. If we go down to 0%, we can target everyone.
As in our previous example, we see that the biggest expected profit lies in targeting the entire population with a probability score of more than 2%. However, maybe we have a very tight budget, or we want to develop a unique campaign for really high potential customers. In this case, we can compare our budget to the curve and point out that targeting customers above the 12% probability ratio is still expected to provide a solid return on investment. After that, we can go through the same process we did before to design this campaign. We identify who these customers are, what influences their purchases, and continue to tailor our marketing campaign to their needs.
It starts and ends with business knowledge
We have seen the possibilities and the value that the expected value model can provide, but I must emphasize again how important it is to have business knowledge to ensure that everything works well. It is important to have a solid understanding of the costs and benefits associated with each possible outcome. It is very important to correctly interpret the results of the model in order to fully understand what levels can be drawn to influence the purchase.
Although it is a complicated method, it is not my intention to sound discouraging to the student who is learning these methods for the first time. On the contrary. I am writing about this to highlight that such methods are no longer reserved for large companies. Small and medium-sized businesses have access to the same data collection and modeling tools, opening the door for anyone who wants to take their business to the next level.
References
Provost, F., and Fawcett, T. Business Data Science: What You Need to Know About Data Mining and Data Analytical Thinking. O'Reilly Media.
All images, unless otherwise noted, belong to the author.
