diff --git a/causal-inference-for-the-brave-and-true/23-Challenges-with-Effect-Heterogeneity-and-Nonlinearity.ipynb b/causal-inference-for-the-brave-and-true/23-Challenges-with-Effect-Heterogeneity-and-Nonlinearity.ipynb index d90db4e..2420df5 100644 --- a/causal-inference-for-the-brave-and-true/23-Challenges-with-Effect-Heterogeneity-and-Nonlinearity.ipynb +++ b/causal-inference-for-the-brave-and-true/23-Challenges-with-Effect-Heterogeneity-and-Nonlinearity.ipynb @@ -6,7 +6,7 @@ "source": [ "# 23 - Challenges with Effect Heterogeneity and Nonlinearity\n", "\n", - "Predicting treatment effects at the unit level is extremely difficult due to the lack of ground truth. Since we only observe one potential outcome $T(t)$, we can't directly estimate it. Rather, we have to rely on target transformations (that can also be viewed as cleverly designed loss function) to estimate conditional treatment effects only in expectation. But that is not the only challenge. Because treatment effects are so slippery, its estimators are often quite noisy. This has huge practical consequences for applications where we want to segment units by their treatment effect, like when we want to do personalized treatment allocation.\n", + "Predicting treatment effects at the unit level is extremely difficult due to the lack of ground truth. Since we only observe one potential outcome $Y(t)$, we can't directly estimate it. Rather, we have to rely on target transformations (that can also be viewed as cleverly designed loss functions) to estimate conditional treatment effects only in expectation. But that is not the only challenge. Because treatment effects are so slippery, their estimators are often quite noisy. This has huge practical consequences for applications where we want to segment units by their treatment effect, like when we want to do personalized treatment allocation.\n", "\n", "We will now see that, sometimes, we can get a better treatment effect segmentation if we don't directly try to estimate CATE, but instead focus on another proxy target, which usually has less variance. A common case when this happens is when the outcome variable of interest $Y$ is binary.\n", "\n", @@ -96,7 +96,7 @@ "source": [ "Here is an incredibly common problem you might face if you find yourself working for a tech company: management wants to boost customer conversion to your product by means of some sort of nudge. For example, they might want to increase the number of app installs by offering a 10 BRL voucher for customers to make in-app purchases. Or offer a free ride the first time you use their ride sharing app. Or decrease transaction fees in the first three months in their investment platform. Since nudges are often expensive, they would love to not have to do it for everyone. Rather, it would be great if we could use the conversion boosting nudge only on those customers who are most sensitive to it.\n", " \n", - "In causal inference terms, you can probably tell by now that this type of business problem falls under the treatment effect heterogeneity (TEH) umbrella. Specifically, you have a costly nudge as the treatment $T$, conversion as the binary outcome $Y$ and customer specific pre-treatment features as $X$. You could then estimate the conditional average treatment effect $E[Y_1 - Y_0|X]$ (or $E[Y'(T)|X]$ if the treatment is continuous)$ with something like Double/Debiased ML and finally target with the nudge only the customers with the highest estimated treatment effect. In business terms, you would be personalizing your conversion strategy. You would be finding a segment of customers with high conversion incrementality and using nudges only in them.\n", + "In causal inference terms, you can probably tell by now that this type of business problem falls under the treatment effect heterogeneity (TEH) umbrella. Specifically, you have a costly nudge as the treatment $T$, conversion as the binary outcome $Y$ and customer specific pre-treatment features as $X$. You could then estimate the conditional average treatment effect $E[Y_1 - Y_0|X]$ (or $E[Y'(T)|X]$ if the treatment is continuous) with something like Double/Debiased ML and finally target with the nudge only the customers with the highest estimated treatment effect. In business terms, you would be personalizing your conversion strategy. You would be finding a segment of customers with high conversion incrementality and using nudges only in them.\n", " \n", "However, there is one complication here that makes the TEH approach not so obvious. The fact that the outcome is binary complicates things considerably. Because this is a bit counter intuitive, I rather show what happens first and then explain why it happens.\n", " \n", @@ -110,15 +110,17 @@ " \n", "$ age \\sim G(10, 4) $\n", " \n", - "$ income \\sim G(20, 2) $\n", + "$ \\widetilde{income} \\sim G(20, 2), \\qquad income = 100\\,\\widetilde{income} $\n", " \n", - "Finally, we will simulate the conversion. For that, we will first **create a latent variable that follows a linear model** with random noise. Importantly, notice that `income` is highly predictive of $Y_{latent}$, but **it does not modify the treatment effect**. Put simply, nudge increases $Y_{latent}$ the same for all levels of `income`. In contrast, `age` only affects $Y_{latent}$ through its interaction with the `nudge` treatment. \n", + "Finally, we will simulate the conversion. For that, we will first **create a latent variable that follows a linear model** with random noise. Importantly, notice that `income` is highly predictive of $Y_{latent}$, but **it does not modify the treatment effect on the latent scale**. Put simply, nudge increases $Y_{latent}$ by the same amount for all levels of `income`. In contrast, `age` affects the latent-scale treatment effect through its interaction with the `nudge` treatment. \n", " \n", "$Y_{latent} \\sim N(-4.5 + 0.001 \\ income + nudge + 0.01 \\ nudge \\ age, 1)$\n", " \n", - "Once we have the $Y_{latent}$, we can simulate `conversion` by setting it to $Y_{latent}>x$. First, let's set `x=0` so that the conversion is roughly 50\\%. That is, on average 50\\% of customers convert to our product.\n", + "Once we have $Y_{latent}$, we can simulate `conversion` by setting it to $Y_{latent}>x$. We set `x=0.1` so that conversion is roughly 50\\%. In the simulated data below, about 52\\% of customers convert to our product.\n", " \n", - "$conversion = 1\\{Y_{latent} > 0\\}$\n", + "$conversion = 1\\{Y_{latent} > 0.1\\}$\n", + "\n", + "It is important to keep the two effect scales separate. The absence of an `income`-by-`nudge` interaction means there is no effect modification by income for $Y_{latent}$. It does **not** imply that income cannot modify the treatment effect on conversion after we apply the threshold. We will return to this distinction below.\n", "\n", "\n" ] @@ -295,9 +297,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The ATE for the latent outcome is pretty straightforward. From our data generating model, we know that this effect should be `1 + avg(age)*0.01`. And since the average age is about 40, this gives us an ATE of about 1.4. Where things get a bit more interesting (and complicated) is in the ATE for conversion. **Because conversion is bounded between 0 and 1, its ATE will not be linear**. Hence, we can't deduce it from an easy formula as we did with the latent outcome (there is a formula, but it is quite complicated). Let's just say the effect is smaller. And this makes sense right? I mean, there is no way the treatment could increase conversion by 1.4 points, simply because conversion can't go beyond 100% . Now, I want you to hold on to that fact because it is going to be crucial in understanding what we will see next. \n", + "The ATE for the latent outcome is pretty straightforward. From our data generating model, we know that this effect should be `1 + avg(age)*0.01`. And since the average age is about 40, this gives us an ATE of about 1.4 latent units. Where things get more interesting is the ATE for conversion. **Because conversion is bounded between 0 and 1, its ATE is a nonlinear transformation of the latent model**. We cannot read it directly from the latent treatment coefficient: below, we derive it using the Normal CDF. In particular, an increase of 1.4 latent units cannot mean an increase of 1.4 probability points, because a conversion probability cannot go beyond 100%. Hold on to this distinction between the latent and probability scales; it will be crucial for what we see next. \n", " \n", - "Let's talk about conditional average treatment effects (CATE) now. Looking at our data generating process, we know for a fact that `estimated_income` predicts conversion but does not modify the effect of the nudge on conversion. Therefore, segmenting our customers based on `estimated_income` will generate segments with the same treatment effect. In contrast, `age` only affects conversion by its interaction with the nudge. So, different age segments will respond very differently to the treatment, while different different income segments won’t. In other words, `estimated_income` should not be a good personalization variable while `age` should be.\n", + "Let's talk about conditional average treatment effects (CATE) now. Looking only at the latent linear model, `estimated_income` predicts the outcome but does not interact with the nudge. Segmenting customers by `estimated_income` should therefore produce the same treatment effect on $Y_{latent}$. In contrast, `age` interacts with the nudge, so different age segments have different latent-scale treatment effects. It is tempting to carry this conclusion over to conversion and expect `age` to be the useful personalization variable. As we will see, applying the threshold makes that conclusion incomplete.\n", " \n", "One way to see this is through the cumulative effect curve. The curve for `age` should start very far from the ATE and slowly converge to it, while the curve for `estimated_income` should only fluctuate around the ATE. This is exactly what we see when we plot the cumulative effect curve for the nudge effect on the latent outcome." ] @@ -381,7 +383,17 @@ "source": [ "However, we also have A LOT of treatment effect heterogeneity by `estimated_income`. Customers with higher `estimated_income` have a much lower treatment effect, which causes the cumulative effect curve to start all the way at zero, to then overshoot the ATE, and only then to converge to it. This tells us that, as far as personalization is concerned, `estimated_income` will generate segments that have more treatment effect heterogeneity (TEH) compared to the segments we would get with `age`. \n", " \n", - "This is inconvenient right? How come the feature we know to drive effect heterogeneity, `age`, is worse for personalization when compared with a feature (`estimated_income`) we know not to modify the treatment effect? The answer lies in the **non-linearity of the outcome function**. Although `estimated_income` does not modify the effect of the nudge on the latent outcome, it does once we transform that latent outcome to conversion (at least indirectly). Conversion is not linear. This means that **its derivative changes depending on where you are**. Since conversion can only go up to 1, if it is already very high, it will be hard to increase it further. In other words, the derivative at high conversion is very low. The same thing is true at the lower end: Because conversion is also bounded below at zero, it will also have a low derivative if it is already very low. Conversion follows an S-shape, with low derivatives at both ends, which we can see by plotting the average conversion by `estimated_income` bins (bins of width 100)." + "This is inconvenient, right? How can `age`, the feature with an explicit latent-scale treatment interaction, look worse for personalization than `estimated_income`, which has no such interaction? The answer lies in the **non-linearity introduced by thresholding the latent outcome**. Let $\\mu_0=-4.5+0.001\\,income$ be the untreated latent mean and $\\delta=1+0.01\\,age$ the nudge's latent-scale effect. Since the latent noise is standard Normal, the conversion probabilities and the conversion CATE are\n", + "\n", + "$$\n", + "P(Y=1\\mid T=t,X)=\\Phi(\\mu_0+t\\delta-0.1),\n", + "$$\n", + "\n", + "$$\n", + "CATE(X)=\\Phi(\\mu_0+\\delta-0.1)-\\Phi(\\mu_0-0.1).\n", + "$$\n", + "\n", + "Even though income does not change $\\delta$, it changes $\\mu_0$, so it changes the treatment effect on the conversion-probability scale. The Normal CDF is S-shaped: its slope is low near probabilities zero and one and highest in the middle. If conversion is already very high, it is hard to increase it further; the same is true when conversion is very low. We can see this shape by plotting average conversion by `estimated_income` bins (bins of width 100)." ] }, { @@ -551,13 +563,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To summarize, what we saw is that when the outcome is binary, the treatment effect tends to be dominated by the curvature (derivative) of the S-shaped function.\n", + "To summarize, in threshold-based binary response models like this one, the outcome-scale treatment effect is strongly shaped by the curvature of the S-shaped response function.\n", " \n", "![image.png](data/img/hte-binary-outcome/logistic.png)\n", " \n", - "For instance, in our conversion problem, if the **average conversion is low**, we are at to the left of the logistic curve and the **treatment effect will be higher at high baseline conversion**. This would translate to a nudge policy that advocates for treating (nudging) those customers with an already high probability of conversion. On the other hand, if the **average conversion is high**, we will be to the right side of the logistic curve, where the derivative (and hence the treatment effect) will be **higher for those customers with lower baseline conversion**. \n", + "For instance, in our conversion problem, if the **average conversion is low**, we are on the left of the S-shaped response curve and the **treatment effect will be higher at high baseline conversion**. This would translate to a nudge policy that advocates for treating (nudging) those customers with an already high probability of conversion. On the other hand, if the **average conversion is high**, we will be on the right side of the response curve, where the derivative (and hence the treatment effect) will be **higher for those customers with lower baseline conversion**. \n", " \n", - "This is certainly a lot to remember, but we can simplify: **just treat whomever is closer to a baseline conversion of 50%**. The mathematical argument here is pretty solid: the derivative of the logistic function is at its peak at 50%, so just treat units closer to that point. \n", + "This is certainly a lot to remember, but the useful heuristic is to **prioritize units whose response lies near the steep middle of the S-curve**, rather than in either tail. For a very small latent-scale shift, the probability change is approximately the derivative times that shift, so it is largest near a baseline conversion of 50%. For a finite positive shift $\\delta$ in our probit model, let $z$ denote the untreated latent index. The exact difference $\\Phi(z+\\delta)-\\Phi(z)$ is maximized at $z=-\\delta/2$: the treatment moves the latent index across the center, and the untreated baseline probability need not be exactly 50%. This principle still depends on the response model, the size of the shift, treatment costs, and other effect modifiers; it is not a universal treatment-allocation rule. \n", " \n", "What is even nicer is that this is one of the rare cases where received wisdom matches the math. In marketing, where conversion problems are very common, there is a belief that we should target neither lost bets (those with very low conversion probability) nor sure wins (those with very high conversion probability). Instead, we should target those in the middle. This is fascinating, since it is the exact same thing we figured using a more formal causal argument. " ] @@ -568,9 +580,9 @@ "source": [ "# Continuous Treatment and Non-linearity\n", " \n", - "We've explored in depth just one example of binary outcome making Heterogeneous Treatment Effect analysis harder. But this phenomenon goes beyond the conversion problem from marketing. For instance, in 2021, the world managed to deliver the first batch of approved COVID-19 vaccines to the general public. Back then, a crucial question was who should receive the vaccine first. This is, not surprisingly, a Heterogeneous Treatment Effect problem. Policy makers would like to vaccinate those who would benefit the most first. In this situation, the treatment effect is averting death or hospitalization. So, whose death or likelihood to be hospitalized would decrease the most when given a shot? In most countries, it was the elderly and those with pre-existing health conditions (comorbidities). These are people that are **more likely to die when infected with COVID-19**. Also, COVID mortality rate is (thankfully!) much lower than 50%, which puts us to the left of logistic function. In this region, by the same argument we made for marketing, it would make sense to treat those with a high baseline probability of death when getting COVID-19, which are precisely the groups we’ve mentioned earlier. Is this a coincidence? Maybe. Keep in mind that I'm not a health expert, so I might be very wrong here. But the logic makes a lot of sense to me. \n", + "We've explored in depth just one example of binary outcome making Heterogeneous Treatment Effect analysis harder. But this phenomenon goes beyond the conversion problem from marketing. For instance, in 2021, the world managed to deliver the first batch of approved COVID-19 vaccines to the general public. Back then, a crucial question was who should receive the vaccine first. This is, not surprisingly, a Heterogeneous Treatment Effect problem. Policy makers would like to vaccinate those who would benefit the most first. In this situation, the treatment effect is averting death or hospitalization. So, whose death or likelihood to be hospitalized would decrease the most when given a shot? In most countries, it was the elderly and those with pre-existing health conditions (comorbidities). These are people that are **more likely to die when infected with COVID-19**. Also, the COVID mortality rate is (thankfully!) much lower than 50%, which places it on the low-probability side of an S-shaped response curve. In this region, by the same argument we made for marketing, it would make sense to treat those with a high baseline probability of death when getting COVID-19, which are precisely the groups we’ve mentioned earlier. Is this a coincidence? Maybe. Keep in mind that I'm not a health expert, so I might be very wrong here. But the logic makes a lot of sense to me. \n", " \n", - "In both cases, marketing nudges and COVID-19 vaccines, **the key complicating factor for Treatment Effect Heterogeneity is the non-linearity of the outcome function** $Y(0)$. This nonlinearity makes it so that, as we go from $Y(0)$ to $Y(1)$, the increase in the outcome is primarily due to the curvature in the outcome function. We saw how this happened with a binary outcome, where $E[Y|X]$ follows a logistic shape. But this is more general. In fact, it is a problem that keeps popping up in business, especially if the treatment is a continuous variable. Let's go through one last example to drive the point home.\n", + "In both cases, marketing nudges and COVID-19 vaccines, **the key complicating factor for Treatment Effect Heterogeneity is a non-linear conditional mean response**. The same latent-scale shift can map to different outcome-scale effects depending on the baseline. In our simulation, $E[Y|T,X]$ follows a probit shape because the latent noise is Normal; a logistic link produces the same qualitative S-shape. But the broader problem goes beyond binary outcomes. In fact, it keeps popping up in business, especially when the treatment is continuous. Let's go through one last example to drive the point home.\n", " \n", "Let's consider the classic pricing problem. You are working for a streaming company like Netflix or HBO. A key question the company wants answered is what price to charge customers. In order to answer the question, they run an experiment in which they randomly assign customers to differently priced deals: 5 BLR/month, 10 BRL/month, 15 BRL/month or 20 BRL/month. By doing so, they hope to answer not just how sensitive customers are to price increases, but also if some types of customers are more sensitive than others. In the plot below, you can see the results from that experiment broken down by two customer segments: `A`, customers with higher estimated income, and `B`, customers with lower estimated income. " ] @@ -611,13 +623,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "With this data, the company wishes to answer the following question: who is more sensitive to discounts? In other words, how can we **rank customers by their sensitivity to price (price elasticity of sales)**? By looking at the curve, we get a feeling that segment `A` is overall less sensitive to discount, even though it generates more revenues. However, we can also see that there is some curvature there. In fact, if we take this curvature into account, the ranking of the treatment effect is no longer just between `A` and `B` customers. The treatment effect will also depend on where they are in the treatment curve. For example, the treatment effect of going from 15 BRL to 10 BRL on customers of segment `A` is higher than the treatment effect of going from 5BRL to 10BRL on customers of segment `B`:\n", + "With this data, the company wishes to answer the following question: who is more sensitive to price changes? In other words, how can we **rank customers by the absolute change in sales caused by a price change**? This is a raw sales response, rather than formal price elasticity, which would scale percentage changes in sales by percentage changes in price. Looking at the curve, segment `A` appears less sensitive overall, even though it generates more sales. However, there is also curvature. Once we account for it, the treatment-effect ranking is no longer just between `A` and `B`; it also depends on the starting price. For example, the absolute effect of increasing price from 10 BRL to 15 BRL for segment `A` is larger than the absolute effect of increasing it from 5 BRL to 10 BRL for segment `B`:\n", " \n", "$$\n", - "E[Y(10) - Y(5) | Seg=B] < E[Y(15) - Y(10) | Seg=A]\n", + "\\left|E[Y(15) - Y(10) \\mid Seg=A]\\right| > \\left|E[Y(10) - Y(5) \\mid Seg=B]\\right|\n", "$$\n", " \n", - "If we where to order the resulting treatment effects for this experiment, it would look something like this:" + "Indeed, the two absolute changes are $|5000-5300|=300$ and $|5000-5100|=100$. If we were to order the resulting absolute treatment effects for this experiment, it would look something like this:" ] }, { @@ -658,7 +670,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Just like in the case where the outcome was binary, in this example, **the treatment effect is correlated with the outcome**. The higher the sales (lower the price), the lower the absolute treatment effect; the lower the sales (higher the price) the lower the absolute treatment effect. But in this case, the situation is even more complicated because the **effect is not only correlated with the outcome, but with the treatment level**. This makes answering counterfactual questions trickier. For example, pretend for a moment that your experimental data actually looks like the following plot, where you test higher prices for the segment `A` (rich population) but only lower prices for the `A` population. This is very common, as firms often want to experiment around the treatment they think is more reasonable." + "Just like in the case where the outcome was binary, in this example, **the treatment effect is correlated with the outcome**. Along either segment's curve, higher sales (at lower prices) correspond to a smaller local absolute treatment effect, while lower sales (at higher prices) correspond to a larger local absolute treatment effect. But in this case, the situation is even more complicated because the **effect is not only correlated with the outcome, but with the treatment level**. This makes answering counterfactual questions trickier. For example, pretend for a moment that your experimental data actually looks like the following plot, where you test higher prices for segment `A` (the richer population) but only lower prices for segment `B`. This is very common, as firms often want to experiment around the treatment they think is more reasonable." ] }, { @@ -697,11 +709,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now, if you naively aggregate the results of the treatment effect, it will look like segment `A` is much more elastic (has higher absolute treatment effect) to price increase than segment `B`. But that is only because, for segment `B`, you've explored the low treatment effect region.\n", + "Now, if you naively aggregate the treatment effects, segment `A` will look much more sensitive to price increases than segment `B`. But that is only because, for segment `B`, you've explored the low-treatment-effect region.\n", " \n", "So what can you do when treatment effects change depending on where you are in terms of both treatment and outcome? To be honest, this is still an active area of research. In practical terms, the best thing you can do is to be **very careful** when trying to answer which type of customer is more sensitive to the treatment. Make sure that the compared customer types all had the same treatment distribution. And, if not, be very skeptical of extrapolating the treatment effect. For instance, in the example above, even though customer `B` looks less sensitive to price increase, you don't know if that will still be the case if you assign higher prices beyond 10BRL to this segment.\n", " \n", - "Another thing you can try to do is to linearize the response curve. The idea here is to get rid of the curvature by transforming the treatment or the outcome (or both) so that the relationship between them looks like a line. Since a line has constant derivative, this would get rid of the problem of the treatment effect changing depending on where you are in the curve. As an example, if we take our price variable and transform it by making it negative, exponentiating it by 4 and reversing the sign, we get a somewhat linear(ish) relationship. In this transformed data, the statement that `A` is less sensitive to price increases then `B` makes much more sense, since it now does not depend on where we are at the curve." + "Another thing you can try to do is to linearize the response curve. The idea here is to get rid of the curvature by transforming the treatment or the outcome (or both) so that the relationship between them looks like a line. Since a line has constant derivative, this would get rid of the problem of the treatment effect changing depending on where you are in the curve. As an example, if we take our price variable and transform it by making it negative, exponentiating it by 4 and reversing the sign, we get a somewhat linear(ish) relationship. In this transformed data, the statement that `A` is less sensitive to price increases than `B` makes much more sense, since it now does not depend on where we are on the curve." ] }, { @@ -741,15 +753,15 @@ "source": [ "## Key Concepts\n", " \n", - "I realize I may have raised more questions than provide answers. Alas, sometimes the best we can do about a problem is to be very aware of it. In this chapter, I hope I've managed to open your eyes to the complications that arise when the outcome we care about is non-linear. \n", + "I realize I may have raised more questions than I have provided answers. Alas, sometimes the best we can do about a problem is to be very aware of it. In this chapter, I hope I've managed to open your eyes to the complications that arise when the outcome we care about is non-linear. \n", " \n", - "This is a common and more studied problem with binary outcomes. In this case, the treatment effect tends to be higher the closer we are to an average outcome of 0.5. Since the outcome is bounded at 0 and 1, effects tend to be very small if we are too close to 0 or 1. \n", + "This is a common and more studied problem with binary outcomes. In common probit and logistic response models, probability-scale treatment effects tend to be larger near the steep middle of the response curve; for small shifts, this is near a baseline outcome of 0.5. Since the outcome is bounded at 0 and 1, effects tend to be very small in either tail. \n", " \n", "Things get more complicated with non-linearities arising in continuous outcome situations. Here, the best you can do is think very carefully about the problem. Try to answer if you care more about treatment effect regardless of treatment baseline or if the baseline is important. That alone will be a valuable guiding principle. \n", " \n", "## Reference\n", " \n", - "Most of the things written here are from my own experience with this problem. However, I did find one academic articles that touch on this subject: *Causal Classification: Treatment Effect Estimation vs. Outcome Prediction*, by Fernández-Loría and Provost talk about the case where treatment effect is correlated with the outcome variable. \n", + "Most of the things written here are from my own experience with this problem. However, I did find one academic article that touches on this subject: *Causal Classification: Treatment Effect Estimation vs. Outcome Prediction*. Fernández-Loría and Provost discuss the case where the treatment effect is correlated with the outcome variable. \n", " \n", "## Contribute\n", " \n",