The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity

Authors:
Peter Ebbes;Dominik Papies;Harald J. van Heerde
Affiliations:
Fisher College of Business, Ohio State University, Columbus, Ohio 43210;Institute for Marketing and Media, University of Hamburg, 20354 Hamburg, Germany;University of Waikato, Hamilton 3240, New Zealand/ and Extramural Fellow at CentER, Tilburg University, 5000 LE Tilburg, The Netherlands
Venue:
Marketing Science
Year:
2011

Citing 7
Cited 0

Logit Demand Estimation Under Competitive Pricing Behavior: An Equilibrium Framework

Management Science
Endogeneity in Brand Choice Models

Management Science
Endogeneity and Heterogeneity in a Probit Demand Model: Estimation Using Aggregate Data

Marketing Science
Unobserved Retailer Behavior in Multimarket Data: Joint Spatial Dependence in Market Shares and Promotion Variables

Marketing Science
Endogeneity in Marketing Decision Models

Marketing Science
Commentary---Assumptions, Explanation, and Prediction in Marketing Science: “It's the Findings, Stupid, Not the Assumptions”

Marketing Science
Commentary---Relevancy Is Robust Prediction, Not Alleged Realism

Marketing Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Market response models based on field-generated data need to address potential endogeneity in the regressors to obtain consistent parameter estimates. Another requirement is that market response models predict well in a holdout sample. With both requirements combined, it may seem reasonable to subject an endogeneity-corrected model to a holdout prediction task, and this is quite common in the academic marketing literature. One may be inclined to expect that the consistent parameter estimates obtained via instrumental variables (IV) estimation predict better than the biased ordinary least squares (OLS) estimates. This paper shows that this expectation is incorrect. That is, if the holdout sample is similar to the estimation sample so that the regressors are endogenous in both samples, holdout sample validation favors regression estimates that are not corrected for endogeneity (i.e., OLS) over estimates that are corrected for endogeneity (i.e., IV estimation). We also discuss ways in which holdout samples may be used sensibly in the presence of endogeneity. A key takeaway is that if consistent parameter estimates are the primary model objective, the model should be validated with an exogenous (rather than endogenous) holdout sample.