Uplift Models for better marketing campaigns (Part 2)
In the part 1 of this blog series, we looked at how D2C(emails, sms, etc.) campaigns work and discussed how uplift models can help in identifying the ‘Persuadables’ from a group of customers. In this post, we will look at how these models are built.
Uplift model is a generic name given to a group of modeling methodologies which serve the same purpose. Uplift model uses previous campaign data to identify high-lift customers and enables us to target them in the next iteration of campaign. With regards to past campaign data, there are a couple of things that we will be needing: List of customers in the Target and Control group, their responses to the campaign (typically 0/1 based on if the customer bought the product) and all the relevant variables describing a customer (Age, income, etc.).
Idea Behind Uplift Models
Our final aim here is to predict the lift for each customer. But, a group of customers at a time can be either in Control or in Treatment sample and hence we can’t directly calculate the lift for a group of customers. But, is that really a problem? Do we really need a customer to be in both groups to calculate lift? We want to find out which group of customers will give high lift and a ‘group’ is defined by the characteristics of the customers in it (age, income, etc.). So, as long as we have customers from different age and income groups in both Control and Treatment sample, we are good to calculate the lift for a group. Eg.: Say, we want to calculate lift for customers with age<25 and income>30,000$. I will select these groups from the two samples, calculate their response rates and their difference is the lift. I don’t really need the same group of customers to be in both samples, but I need customers with same characteristics to be in both the samples.
Now, let’s say we have only two variables: age and income. We want to find high-lift groups based on these two variables. We can create 4 bins in both the variables. Eg.: Age: <25, 25 to 35, 35 to 45 and >45. Using the two binned variables we can create totally 4×4=16 groups. For each of these groups, we can calculate the lift and easily find high-lift groups. This high-lift group and other customers with same characteristics who weren’t targeted last time (or were targeted but didn’t convert) will form our Target group for next campaign. I don’t know if it dawned on you, but what we built here is nothing but an uplift model, albeit a very ad hoc one. It tells you which are high lift customers based only on age & income and who should be targeted. But, the same methodology cannot be adopted when we have, say, 50 customer-characteristics related variables.
Uplift Model: Two Model Approach
The Two Model approach is easiest to understand and apply. Here, we simply build two classification models, one on treatment sample and other on control sample. These models tell us the likelihood of a customer to respond when in the respective samples.
After these two models are built, you score all customers and have two probabilities per customer: St and Sc as explained in the image. Obviously, these are based on the characteristics that went into building these models.
Now, comes the interesting part. You know that St and Sc are the likelihood of a customer responding when in Treatment and Control respectively. Hence, St-Sc is nothing but the expected lift of a customer! Once we know the expected lift, it is pretty easy to split the customers into four groups that we had talked about in the last post. Of these four, Persuadables can be identified by high St-Sc.
Although it might seem trivial to build uplift models like this, it’s nowhere close to that. Uplift models need large amount of data to ensure that the individual models are not noisy. Using difference of probabilities to predict lift, will lead to very high errors if errors from the two probabilities add up. Hence, it’s important to build models which aren’t overfitting. Feature selection can make or break an uplift model based on how noisy they are. We will talk about this in the next post and for sake of continuity move directly to model validation.
Model Validation
If you have understood what we did till now, model validation will be cakewalk for you. To validate the model, you need to split all your customers into, say, 10 bins based on their St-Sc. Bin 1 is the bin with highest St-Sc and Bin 10 with lowest. These bins will contain some customers who were in Treatment sample and some in Control. You calculate the Treatment Response rate(Rt) and Control response rate(Rc) in each bin. Rt-Rc is your actual lift in that bin and St-Sc is the predicted lift. Now, plot the Rt-Rc for each bin and check if there’s any trend. Following is an example:
The green line is the campaign’s overall lift which is around 3% and red line is the actual lift in each bin. The top two bins can give us customers whose lift is two times the overall lift. The Bins with high lift(1 to 7) can be said to contain the Persuadables. The bins with negative lift(10) contain the Do-not-disturbs and other bins(8,9) contain Sure-things or Lost-Causes.
A metric called qini tells the power of your uplift model. It is similar to gini which is nothing but scaled version of AUC. You might want to look it up.
In the next and last post on uplift models, we will talk about feature selection and some things to keep in mind while building models for this purpose.
