Accounting for Causality When Measuring Sales Lift from Television Advertising

Experimentation versus Observational Data

Experimentation is the gold standard in measuring the causal effects of advertising on sales by randomly assigning respondents to an exposure group and a nonexposure group, thus ensuring that the response (sales) is a function of the experimental stimulus (advertising; e.g., Aaker and Carman, 1982; Hu, Lodish, and Krieger, 2007; Lodish, Abraham, Kalmenson, Livelsberger, et al., 1995). Experimentation has been used previously to test pre-campaign advertising effects through test markets (Eskin, 1975; Eskin and Baron, 1977). Experimentation is difficult to implement once a campaign is launched, because of the difficulty in ensuring the integrity of a control (nonexposure) group. Marketers using traditional media such as television and radio would find it difficult to ensure that a control group was not exposed to any advertising during the campaign period. They could utilize split-cable television, but directing resources is more likely to be a function of marketing rather than experimental objectives; namely, targeting the most likely buyers rather than assigning respondents to randomized groups. Marketers may be reluctant to withhold advertising from a randomized control group because of the sacrifice in revenue as a result. If test groups are selected by targeting criteria rather than randomization, this creates the same problems of endogeneity as for observational data; namely, that the relationship between advertising and sales could be artifactual in that sales results are independent of advertising exposure.

The primary alternative to experimentation is to use observational data (e.g., single-source data relating advertising exposure to purchases at the individual level) and match those not exposed with those exposed on a variety of criteria (e.g., Bronnenberg, Dubé, and Mela, 2010; Draganska, Hartmann, and Stanglein, 2014; Kumar, Bezawada, Rishika, and Janakiraman, et al., 2016). The problem here is to ensure that matching treatment and control groups on key criteria is sufficiently randomized to control for endogeneity. This problem has been reported in previous research (Blake, Nosko, and Tadelis, 2015; Lewis, Rao, and Riley, 2011).

Measuring Sales Lift

The authors define sales lift as the difference in sales results before and immediately after an advertising campaign resulting from the campaign. Berkovich and Wood (2016) explained that, to be included in a sales lift study, households must meet pre- and post-campaign periods “static” to ensure that they are actively purchasing the focal product. This static is a function of various shopping behaviors, such as purchase frequency, purchase spending, and interpurchase time. Similarly, the current authors measure sales lift by using different household samples as a function of purchase frequency and purchase spending on the focal brand and category during the pre- and post-campaign periods (e.g., top 10 percent, 25 percent, and 50 percent of total households in terms of purchase frequency and spending for the focal brand). In this study, the pre-campaign period is defined as one year before the start of the television campaign. Although Berkovich and Wood (2016) measured household-level incremental sales considering matched samples between test and control groups, they primarily relied on demographics and purchase statistics in a matching process and then simply compared statistics of purchase behaviors between those groups to measure sales lift. In contrast, this study exploits formal econometric approaches—propensity score matching and difference in differences—for matching samples and measuring sales lift, and will discuss all details of the empirical methods in the subsequent section.

Data and Summary Statistics

This study tests sales lift for a brand in a chocolate-candy category for a television campaign that ran from March 28 to June 16, 2016. Sales data refer to household-based scanner data collected by Nielsen Catalina Solutions for a sample of households in the United States whose purchases are scanned and recorded after each shopping trip. The data are single source also, combining television advertising exposure and purchase data from the same household at the individual level so that the effects of television advertising on purchase behaviors can be studied. There are data for 435,327 households in total, including all shopping information from March 29, 2015, to June 26, 2016. (See Table 1 for the summary statistics for all the details of purchase behavior in the chocolate-candy product category.)

Controlling for Endogeneity

Because this study uses observational data, controlling for endogeneity is central to the analysis. Two sources of endogeneity in the relationship between advertising and sales are targeting-induced endogeneity and exposure-induced endogeneity (Gordon, Zettelmeyer, Bhargava, and Chapsky, 2019; Lewis et al., 2011). Targeting-induced endogeneity occurs when marketers target heavy buyers, resulting in purchases that would have occurred independent of advertising effects. Relating advertising to sales overstates the association. Exposure-induced endogeneity is a function of variations in advertising exposure independent of advertising effects. If demographic characteristics are related to the likelihood of viewing a campaign, exposure is likely to be a function of such characteristics as well as advertising effectiveness. Relating advertising effectiveness to sales must control for demographic covariates related to the level of advertising exposure.

Additionally, previous literature (e.g., Anand and Shachar, 2011; Lovett and Peress, 2015) particularly consider endogeneity of television advertising that is mainly driven by unobserved characteristics such as geography or demographics. The most recent study investigated the problem of individual-level television advertisement targeting by leveraging granular data linking household-level television advertisement viewing with product purchase (Tuchman, Nair, and Gardete, 2018). Although these papers rigorously control for advertising endogeneity using a structural model, it would be difficult for practitioners to implement and replicate this strategy because of a sophisticated analysis process. Therefore, the authors propose another approach to control for television advertising endogeneity in a reduced-form way that is simpler and easier to execute.

The authors combine two state-of-the art econometric approaches, difference in differences and propensity score matching, to measure television advertising effects after controlling for the endogeneity of television advertising exposure. Ever since the work of Ashenfelter and Card (1985), the use of the difference in differences method has become widespread in marketing and economics (e.g., Angrist and Pischke, 2009; Goldfarb and Tucker, 2014; Imbens and Wooldridge, 2009; Meyer, 1995). In this approach, outcomes (i.e., purchases of the focal brand) are observed for two groups (the advertisement-exposed group and the non-advertisement-exposed group) for two time periods (pre- and post-campaign periods).

Specifically, the following regression equation was used: (1)

where i indicates a household and t indicates a shopping occasion. Purchase_it is an indicator equal to 1 if household i purchased the focal brand at time t. Similarly, Ad Exposure_i is an indicator equal to 1 if household i was exposed to a television advertisement at least once during the campaign period and 0 if otherwise. Campaign Period_t is also an indicator equal to 1 if time t is after the starting date of the television advertising campaign; otherwise, it is equal to 0. The final variable is Price_it. Last, ε_it is a stochastic error term.

First, price is one of the most important components affecting consumer purchase in consumer-packaged goods, and it is expected to influence purchase response negatively. Second, to identify the effect of television advertisements on purchase, Ad Exposurei, Campaign Periodt, and the interaction term must be measured. One of the groups is exposed to television advertisements in the campaign period but not in the pre-campaign period. The second group is not exposed to television advertisements during either period. In the case in which the same units within a group are observed in each time period, the average gain in the second (control) group can be subtracted from the average gain in the first (treatment) group. This removes biases in the second period comparisons between the treatment and control groups that could be the results of permanent differences between those groups, as well as biases from comparisons over time in the treatment group that could be the results of trends.

As for the interpretation of the coefficients, β₁ captures possible differences between the television-advertising-exposed and non-advertising-exposed groups. The coefficient of the time period, dummy β₂, captures aggregate factors that would affect changes in the purchase of our focal brand, even in the absence of the television advertising campaign. The authors aim to determine coefficient β₃, which accounts for the interaction term between treatment status and behavior in the pretreatment period. (2)

β₃ measures the incremental advertising effects of those exposed to the campaign versus those not exposed while accounting for pre-campaign behavior.

This coefficient measures the causal effects of television advertising exposure during the campaign period on brand purchases. The authors hypothesize that β₃ is positive and significant if television advertisements have positive causal effects on brand purchases. The authors run the ordinary least squares regression of the equation, a linear probability model.

As mentioned previously, however, the advertising exposure variable in Equation 1 is endogenous because of the exposure selection issue. To control for the endogeneity of television advertising exposure, this paper uses another state-of-the-art econometric technique: propensity score matching. This approach is used in many academic works in marketing; for example, examining the effectiveness of Internet and television advertisements on brand building (Draganska et al., 2014), understanding the effects of firm-generated social media on customer metrics (Kumar et al., 2016), and investigating the effects of access to digital video recorders on sales of advertised products (Bronnenberg et al., 2010). The propensity score is the probability of treatment assignment (i.e., status of being exposed to television advertisements) conditional on observed baseline characteristics (i.e., demographics and precampaign period purchase behaviors). The propensity score allows the design and analysis of an observational (nonrandomized) study so that it mimics some of the particular characteristics of a randomized controlled trial. In particular, the propensity score is a balancing score so, conditional on the propensity score, the distribution of the observed baseline covariates will be similar for advertising-exposed and non-advertising-exposed households.

To compute the propensity score, the authors used the following equation: (3)

where i indicates a household and Ad Exposurei is an indicator equal to 1 if household i was exposed to television advertisements at least once during the campaign period and 0 otherwise. The important term is X_i, which is a vector of covariates, demographics, and pre-campaign period purchase behaviors (including income, age, household size, race, gender, population of the household’s location, marital status, education, origin, purchase spending, purchase amount, purchase with promotion at both the brand and category levels, and main shopping retailers).

On the basis of the propensity scores, the nearest neighbor is matched, enabling matching advertising-exposed households with non-advertising-exposed households so that the two groups can be compared in a process similar to that of a randomized control trial.

The authors found a close match between households exposed and those not exposed to advertising in demographic characteristics and a lack of statistically significant difference in pre-campaign purchase behavior as covariates, which might affect the probability of being exposed to advertisements (See Table 2). A balanced distribution of the covariates affecting advertising exposure probability between these two groups shows how we control for the endogeneity of television advertising exposure.

This approach is similar to the one used by Berkovich and Wood (2016) to match test and control households on the basis of household demographics and shopping characteristics. They stated that the purpose of matching is to be able to attribute any increase in spending to advertising rather than to an existing preference for the product being studied. The current study goes further by matching the households on the basis of their propensity scores of exposure to television advertisements, which is a function of demographics and pre-campaign period purchase behaviors. Specifically, the current study computes the propensity scores of all households in the data and chooses one of the households in the control group (i.e., a household not exposed to television advertising) that shows the nearest propensity score of the household in the treatment group (i.e., a household exposed to television advertising).

After the matching process, there is now a dummy variable, advertising exposure, which is equal to 1 if household i is exposed to advertisements and 0 otherwise for all households. This newly created advertising exposure variable is used as a covariate in Equation 1. By doing so, the effects of the television advertising campaign on brand purchase after controlling for the endogeneity of advertising exposure can be measured (See Table 4, shown later).

Heterogeneous Advertising Effects

The estimates in previous sections (e.g., β₃ in Equation 1) indicate average causal effects of advertising on brand purchase, implying that all the households exposed to the advertising campaign have the same sensitivity to television advertisements. How the effects vary across subhouseholds is also of interest, since different subgroups of the households are likely to react differently to television advertisements. Previous literature in marketing and economics (e.g., Berkovich and Wood, 2016; Rossi and Allenby, 1993; Zenetti and Otter, 2014) emphasizes the importance of such heterogeneous advertising effects on consumer purchase decisions.

To capture heterogeneous advertising effects, this study exploits a more robust analysis that is an extension of the difference in differences analysis described earlier: a difference in difference in differences analysis using a different control group within the treatment state (i.e., advertisement exposure). The causal effects of television advertisements (i.e., β₃ in Equation 1) in the difference in differences analysis imply that all households that were exposed to advertisements have the same sensitivity in responding to advertisements. The effectiveness of the advertisement, however, could vary across households, depending on their purchase behaviors. Heavy brand purchasers, for example, may be more (or less) likely to respond to advertisements compared with light brand purchasers. To understand heterogeneous causal advertising effects on brand purchase, more robust indicator variables are needed: the top 50 percent, top 25 percent, and top 10 percent of brand purchasers based on the total purchase frequency before the campaign period. The indicator variable for the top 50 percent is equal to 1 if the household purchased the study’s focal brand at least twice in the pre-campaign period and 0 if otherwise. The authors create the other indicators in the same manner; for example, the top 25 percent of brand purchasers equals 1 if they bought the brand at least three times in the pre-campaign period and 0 if otherwise. (See Table 3 for a summary of the definition, total number, and actual percentage of these household segments.)

Next, several interaction terms are added in Equation 1 as follows, using the top 50 percent as an example: (4)

As for the interpretation of the coefficients, β₄ captures the causal effects of the campaign on households that have never purchased or purchased at least once the focal brand in the pre-campaign period (i.e., households that are not in the top 50 percent). β₇ denotes the difference in causal effects of the advertising campaign between households that were exposed to the campaign and purchased the focal brand at least twice in the pre-campaign period versus those that did not purchase or purchased at least once (β₄) while controlling for prepurchase behavior. Similarly, replacing the Top50% with the other indicator variables, such as Top25% and Top10%, allows for the heterogeneity of advertising effects between corresponding groups to be calculated. This approach is called the difference in differences in differences. Here, β₇ is described as follows: (5)

Again, β₇ measures the difference in causal effects of television advertisements between the top 50 percent of households (purchased at least twice) and everyone else.

Limitations and Future Research

The data are derived from a single brand source. There is no information on switching behavior and, as a result, the authors cannot analyze competitive influences on the advertising-to-sales relationship in a dynamic marketplace. This means that, although it is possible to measure causal advertising effects on sales for the focal brand, brand managers do not know where a sales increment came from as a result of the advertising campaign. Future research could build on this analysis by investigating a multibrand data source that could show brand managers how consumers substitute their brand purchases due to an advertising campaign. Examining the causal effects that advertisements have on sales by considering a substitution pattern between brands would provide better insight into the product category. Moreover, this analysis focused on one category. Future research could focus on causal advertising effects across categories.

Conclusion

Controlling for advertising endogeneity and accounting for heterogeneity in the campaign for the focal brand, this analysis confirms several findings. First and most important, without matching for pre-campaign purchases, advertising effects in the campaign period become insignificant. This demonstrates the importance of matching by purchase predispositions to account for advertising effects in evaluating the advertising-to-sales relationship. Second, the campaign was effective in influencing both brand purchasers and nonbrand purchasers but more effective for nonbrand purchasers and lighter brand purchasers compared with heavy brand purchasers. Third, without a matching process to control for endogeneity, the analysis on heterogeneous advertisement effects shows incorrect insight (i.e., advertising to heavier brand purchasers is more effective). This demonstrates why using the proposed approaches is important.

The analytical approach illustrated for controlling for endogeneity in accounting for the causal effects of advertising and showing its heterogeneous effects is, perhaps, more significant.