Gleaning Information for Cognitive Operations from Don’t Know Responses in Cognitive and Noncognitive Assessments

Abstract

Abstract The Don’t Know (DK) response – taking the form of an omitted response or not-reached at the end of a cognitive test, or explicitly presented as a response option in a social survey – contains important information that is often overlooked. Direct psychometric modeling efforts for DK responses are few and far between. In this article, the linear logistic test model (LLTM) is proposed for delineating the impacts of cognitive operations for a test that contains DK responses. We assume that the DK response is a valid response. The assumption is reasonable for many situations, including low-stakes cognitive tests and attitudinal assessments. By extracting information embedded in the DK response, the method shows how DK can inform the latent construct of interest and the cognitive operations underlying the response to stimuli. Using a proven recoding scheme, the LLTM could be implemented through commonly used programs such as PROC GLIMMIX. Two simulation experiments to evaluate how well the parameters can be recovered were conducted. In addition, two real data examples, from a noncognitive test of health belief assessment and a cognitive test of knowledge in diabetes, are also presented as case studies to illustrate the LLTM for DK response.