Gremlins in the Data: Identifying the Information Content of Research Subjects

Keywords

conjoint analysis, data quality, finite mixture models, multinomial logit

Abstract

Empirical demand functions, such as those from choice-based conjoint analyses, are critical to many aspects of marketing. Approaches have been developed to ensure that research subjects provide honest and thoughtful responses. However, to reduce costs, researchers increasingly collect data online, under conditions that compromise the value of the information provided. Objective measures related to how the study is completed, such as latency (how quickly answers are given), can only be tied to other objective measures (such as the consistency of the answers), but ultimately their relationship to the subject’s utility function is questionable. To address this problem, the authors introduce a mixture modeling framework that clusters subjects based on variances. The proposed model naturally groups subjects based on their internal consistency. The authors argue that a higher level of internal consistency (i.e., lower variance) reflects more engaged consumers who have sufficient experience with the product category and choice task. “Gremlins,” in contrast, behave such that the noise in their responses overwhelms any signal, leading to a lack of predictive power. This approach provides an automated way to determine which respondents are relevant. The authors discuss the conceptual and modeling framework and illustrate the method using both simulated and commercial data.

Original Publication Citation

John R. Howell, Peter Ebbes, and John Leichty, "Gremlins in the Data: Identifying the Information Content of Research Subjects", Journal of Marketing Research, 2021.

Document Type

Peer-Reviewed Article

Publication Date

2021

Publisher

Journal of Marketing Research

Language

English

College

Marriott School of Business

Department

Marketing

University Standing at Time of Publication

Assistant Professor

Link to Full Text

Share

COinS