Frederick G. Conrad

Does Conversational Interviewing Reduce Survey Measurement Error

Standardized survey interviewing is widely advocated in order to reduce interviewer-related error, for example by F. L. Fowler and T. W. Mangione. But L. Suchman and B. Jordan argue that standardized wording may decrease response accuracy because it prevents the conversational flexibility that respondents need in order to understand questions as survey designers intended. The authors propose that the arguments for these competing positions - standardized versus flexible interviewing approaches - may be correct under different circumstances. In particular, both standardized and flexible interviewing should produce high levels of accuracy when respondents have no doubts about how concepts in a question map onto their circumstances. However, flexible interviewing should produce higher response accuracy in cases where respondents are unsure about these mappings. The authors demonstrate this in a laboratory experiment in which professional telephone interviewers, using either standardized or flexible interviewing techniques, asked respondents questions from three large government surveys. Respondents answered on the basis of fictional descriptions so that the authors could measure response accuracy. The two interviewing techniques led to virtually perfect accuracy when the concepts in the questions clearly mapped onto the fictional situations. When the mapping was less clear, flexible interviewing increased accuracy by almost 60 percent. This was true whether flexible respondents had requested help from interviewers or interviewers had intervened without being asked for help. But the improvement in accuracy came at a substantial cost - a large increase in interview duration. They propose that different circumstances may justify the use of either interviewing technique

Clarifying Question Meaning in a Household Telephone Survey

This study contrasts two interviewing techniques that reflect different tacit assumptions about communication. In one, strictly standardized interviewing, interviewers leave the interpretation of questions up to respondents. In the other, conversational interviewing, interviewers say whatever it takes to make sure that questions are interpreted uniformly and as intended. Respondents from a national sample were interviewed twice. Each time they were asked the same factual questions from ongoing government surveys, five about housing and five about recent purchases. The first interview was strictly standardized; the second was standardized for half the respondents and conversational for the others. Respondents in a second conversational interview answered differently than in the first interview more often, and for reasons that conformed more closely to official definitions, than respondents in a second standardized interview. This suggests that conversational interviewing improved comprehension, although it also lengthened interviews. We conclude that respondents in a national sample may misinterpret certain questions frequently enough to compromise data quality and that such misunderstandings cannot easily be eliminated by pretesting and rewording questions alone. More standardized comprehension may require less standardized interviewer behavior.

What they see is what we get: response options for web surveys

Several alternative response formats are available to the web survey designer, but the choice of format is often made with little consideration of measurement error. The authors experimentally explore three common response formats used in web surveys: a series of radio buttons, a drop box with none of the options initially displayed until the respondent clicks on the box, and a scrollable drop box with some of the options initially visible, requiring the respondent to scroll to see the remainder of the options. The authors reversed the order of the response options for half the sample. The authors find evidence of response order effects but stronger evidence that visible response options are endorsed more frequently, suggesting that visibility may be a more powerful effect than primacy in web surveys. The results suggest that the response format used in web surveys does affect the choices made by respondents.

Evaluating the Effectiveness of Visual Analog Scales

The use of visual analog scales (VAS) in survey research has been relatively rare, in part because of operational difficulties. However web surveys permit the use of continuous input devices such as slider bars, making VAS more feasible. The authors conducted an experiment to explore the utility of a VAS in a web survey, comparing it to radio button input and numeric entry in a text box on a series of bipolar questions eliciting views on genetic versus environmental causes of various behaviors. The experiment included a variety of additional comparisons including the presence or absence of numeric feedback in theVAS, the use of a midpoint or no midpoint for the other two versions, and numbered versus unnumbered radio button scales. The response distributions for theVAS did not differ from those using the other scale types, and theVAS had higher rates of missing data and longer completion times.

A heuristic evaluation of a World Wide Web prototype

The authors describe their experience with a method, heuristic evaluation, for testing the usability of a World Wide Web site design. Heuristic evaluation is one of the less formal methods of usability testing, and promises to be cheap, easy, and fast. For the most part the authors found this to be true, with some caveats.

Living in History How War, Terrorism, and Natural Disaster Affect the Organization of Autobiographical Memory

Memories of war, terrorism, and natural disaster play a critical role in the construction of group identity and the persistence of group conflict. Here, we argue that personal memory and knowledge of the collective past become entwined only when public events have a direct, forceful, and prolonged impact on a population. Support for this position comes from a cross-national study in which participants thought aloud as they dated mundane autobiographical events. We found that Bosnians often mentioned their civil war and that Izmit Turks made frequent reference to the 1999 earthquake in their country. In contrast, public events were rarely mentioned by Serbs, Montenegrins, Ankara Turks, Canadians, Danes, or Israelis. Surprisingly, historical references were absent from (post–September 11) protocols collected in New York City and elsewhere in the United States. Taken together, these findings indicate that it is personal significance, not historical importance, that determines whether public events play a role in organizing autobiographical memory.

Usability testing of world wide web sites

World Wide Web (Web) site usability - for better or worse - affects millions of users on a daily basis. As the capabilities of the Web continue to expand through HTML extensions, VRML, Java, and ActiveX, site designers are becoming overwhelmed by a proliferation of viable interaction techniques. Just because a technology is possible, however, does not mean it is desirable, nor that it is being incorporated in a productive manner. Many Web sites, both those connected to the Internet and those available through an institutional intranet, have become cluttered with useless and confusing, albeit cool, new features.

The Design of Grids in Web Surveys

Grid or matrix questions are associated with a number of problems in web surveys. In this article, we present results from two experiments testing the design of grid questions to reduce breakoffs, missing data, and satisficing. The first examines dynamic elements to help guide respondent through the grid, and on splitting a larger grid into component pieces. The second manipulates the visual complexity of the grid and on simplifying the grid. We find that using dynamic feedback to guide respondents through a multiquestion grid helps reduce missing data. Splitting the grids into component questions further reduces missing data and motivated underreporting. The visual complexity of the grid appeared to have little effect on performance.

Disability, participation, and subjective wellbeing among older couples

This paper investigates the link between disability and subjective wellbeing, using data from the 2009 Disability and Use of Time supplement to the Panel Study of Income Dynamics, the longest running national panel study in the United States. Disability is construed broadly to include both the presence of any physical, cognitive, or sensory impairment or activity limitation and also the severity of underlying impairments. Subjective wellbeing is measured using two distinct approaches: reports of life satisfaction and of moment-to-moment wellbeing-both positive and negative-on the previous day. The latter, collected through 24-h time diaries, also offers for the first time the ability to explore the role of participation in particular kinds of activities linking disability to subjective wellbeing. The analytic sample included married persons ages 60 and older and their spouses (n = 751 married individuals) who completed 1498 diaries. Several new findings emerged: no matter what the measure of wellbeing, older married adults with disability report worse subjective wellbeing than those without, and neither different demographic and socioeconomic profiles nor differences in participation fully account for these disparities. Influences of disability on global life satisfaction and episodic reports of happiness were relatively small and of comparable size. However, notably sizeable differences were identified in the cumulative number of pleasant minutes experienced yesterday by disability status - on the order of 71 fewer minutes on average for those with a disability of average severity. Differences appear to be more strongly linked to somatic symptoms of pain and feeling tired than to differential intensity of experiencing happiness, sadness, frustration, or worry. We also found limited support for the notion that participation partially mediates the relationship between disability and global, but not episodic, subjective wellbeing.

Early Appraisals of Electronic Voting

With the recent troubles in U.S. elections, there has been a nationwide push to update voting systems. States and localities are investing heavily in electronic voting systems, many of which use a touch screen. These systems offer the promise of faster and more accurate voting; however, the current reality is that they have some shortcomings in terms of voter usability. This study examines issues related to the usability of electronic voting systems and reports on a series of usability studies that involved expert review, close observation, a field test, and an exit poll to learn voters’responses to a new voting system. Our analysis shows these systems work well; however, they have some shortcomings including some that have raised concerns among a minority of voters.