The Impact of Temporally Turning off TV Ad on Search: A Generalized Synthetic Control Estimator under Interference

Abstract

We measure the effects of major TV advertisers’ temporary discontinuation of TV advertising on consumer keyword search over time, by leveraging a field experiment in the U.S. wireless industry in which the focal brand stopped its TV advertising for one randomly chosen week. The identification challenge lies in the spillover effects of the intervention on other control brands, that are used to eliminate unobserved confounding factors over time: this is complicated because the presence of interference violates the fundamental assumption required for existing causal inference methods. We address this identification issue, by developing a generalized synthetic control method that can debiase interfering “treated” and “control” units when conducting counterfactual analysis. Through a simulation study, we first show that under interference, our proposed method can significantly reduce bias in estimating the true intervention effect compared with existing methods. Applying the proposed method in our empirical context, we find that the focal brand’s intervention resulted in a significant 5% reduction of its daily search volume. This effect initially occurred after two days, but once regular TV advertising was resumed, it took two weeks for search volume to return to the prior level. In addition, we find large treatment heterogeneity across topical search categories. For instance, while search traffic was more likely to be reduced (up to 30% per day) for searches that are relevant to the content of TV ads, this effect wore off quickly. Lastly, we find a positive spillover effect of the intervention on the focal brand’s competitors, which, however, is usually very short-lived.