Mario Callegaro

Where Am I? A Meta-Analysis of Experiments on the Effects of Progress Indicators for Web Surveys

The use of progress indicators seems to be standard in many online surveys. Researchers include them in surveys in the hope they will help reduce drop-off rates. However, there is no consensus in the literature regarding their effects. In this meta-analysis, we analyzed 32 randomized experiments comparing drop-off rates of an experimental group who completed an online survey in which a progress indicator was shown to drop-off rates of a control group to whom the progress indicator was not shown. In all the studies, a drop-off was defined as a discontinuance of the survey (at any point) after it has begun, resulting in failure to complete the survey. Three types of progress indicators were analyzed: constant, fast-to-slow, and slow-to-fast. Our results show that, overall, using a constant progress indicator does not significantly help reduce drop-offs and that effectiveness of the progress indicator varies depending on the speed of indicator: Fast-to-slow indicators reduced drop-offs, whereas slow-to-fast indicators increased drop-offs. We also found that among the studies in which a small incentive was promised, showing a constant progress indicator increased the drop-off rate. These findings question the common belief that progress indicators help reduce drop-off rates.

N the Network

Despite higher hit rates for cell phone samples, inefficiencies in processing calls to these numbers relative to landline numbers continue to be documented in the U.S. literature. In this study, we propose one method for using cell phone provider information and Internet resources for validating number status. Specifically, we describe how we used ‘‘in network’’ options available from three major providers’ web sites to determine the validity of cell phone numbers. We tested differences in working number rates (WNRs) among valid and nonvalid numbers against a normal processing control group and determined that the WNR among valid numbers was approximately 14 percentage points higher than the WNR of the comparison group. This process also shows promise in reducing the effort required to determine working status and may provide a basis for developing screening tools for cell phones that capitalize on resources that are unique to this technology.

IVR and web administration in structured interviews utilizing rating scales: exploring the role of motivation as a moderator to mode effects

Survey researchers have reported differing results on frequency distributions when the same item is delivered via an interactive voice response (IVR) system versus the web. The current paper expands such research into the organizational research field and evaluated the hypothesis that respondent motivation affects the occurrence of mode differences. In this study, personnel selection instruments using five‐point Likert scales were administered to job applicants and job incumbents. Data were collected via IVR or via the web. With job incumbents, the mode effect observed was similar in magnitude to that observed in the survey research literature. However, with job applicants the mode effect was smaller.

Metrics and Design Tool for Building and Evaluating Probability-Based Online Panels

Probability-based online panels are beginning to replace traditional survey modes for existing established surveys in Europe and the United States. In light of this, current standards for panel response rate calculations are herein reviewed. To populate these panels cost-effectively, more diverse recruitment methods, such as, mail, telephone, and recruitment modules added to existing surveys are being used, either alone or in combinations. This results in panel member cohorts from different modes complicating panel response rate calculations. Also, as a panel ages with inevitable attrition, multiple cohorts result from panel refreshment and growth strategies. Formulas are presented to illustrate how to handle multiple cohorts for panel metrics. Additionally, drawing on relevant metrics used for a panel response rate, we further demonstrate a computational tool to assist planners in building a probability-based panel. This provides a means to estimate the recruitment effort required to build a panel of a predetermined size.

Recent Books and Journals Articles in Public Opinion, Survey Methods, Survey Statistics, Big Data and User Experience Research. 2017 Update

Welcome to the 10th edition of this column on recent books and journal articles in the field of public opinion, survey methods, survey statistics, Big Data, data science and user experience research. Yes, it is the 10th anniversary of this paper series started in March 2009. In the first [article](https://www.surveypractice.org/article/2966-recent-books-in-public-opinion-and-survey-methods), there were only books in public opinion and survey methods, but over the years, I added more and more topics related to survey methods given our science is getting more interdisciplinary as ever. Special issues of journals have a space too because in my view, they are like edited books. Finally, I also added review papers from the journal series of _Annual Reviews_ because these papers are seminal state of the art write-ups, a mini book, if you wish on a specific subject. I hope the readers enjoyed the articles over the years and were able to find interesting books to improve their knowledge on a particular subject.

Recent Books and Journals Articles in Public Opinion, Survey Methods, Survey Statistics, and Big Data. 2016 Update

Welcome to the 8th edition of this column on recent books and journal articles in the field of public opinion, survey methods, survey statistics, and Big Data. This year I officially added Big Data to the title as there is very strong interest on the topic and surveys and Big data are becoming more and more interrelated. This article is an update of the April 2015 article . Like the previous year, the books are organized by topic; this should help the readers to focus on their interests. It is unlikely to list all new books in the field; I did my best scouting different resources and websites, but I take full responsibility for any omission. The list is also focusing only on books published in English language and available for purchase (as ebook or in print) at the time of this review (August 2016). Books are listed based on the relevance to the topic, and no judgment is made in terms of quality of the content. We let the readers do so.

Recent Books and Journals Articles in Public Opinion, Survey Methods, and Survey Statistics. 2015 Update

Recent Books and Journals Articles in Public Opinion, Survey Methods, and Survey Statistics. 2015 Update

Survey Practice Book List 2012: Recent Books and Journals in Public Opinion, Survey Methods, and Survey Statistics

This article is an update of the April 2012 article. As the previous year, we organized the books by topics; this should help the readers to focus on their interests. It is unlikely to list all new books in the field; we did our best scouting different resources and websites. The list is also focusing only on books published in the English language and available for purchase at the time of this review (March 2013). Books are listed based on the relevance to the topic and no judgment is made in terms of quality of the content. We let the reader do so. The journal section signals special issues of interest to Survey Practice readers. Finally, this year we started a thematic section with new books devoted to Infographics given the interest in the industry as a tool for visualizing data. If you want to send information for the next issue, please send it to surveypractice.new.books@gmail.com

Recent Books in Public Opinion and Survey Methods

Bell, R.M. and M.L. Cohen, eds. 2008. Coverage measurement in the 2010 census. National Academies Press, Washington, DC. Belli, R.F., F.P. Stafford and D. F. Alwin, eds. 2009. Calendar and time diary methods in life course research. Sage, Los Angeles. Brace, I. 2008. Questionnaire design: how to plan, structure and write survey material for effective market research. Kogan Page, Philadelphia. Brown, L.D. M.L. Cohen and D.L. Cork. eds. 2008. Experimentation and evaluation plans for the 2010 census: interim report. National Academies Press, Washington, DC. Carballo, M. and U. Hjelmar, eds. 2008. Public opinion polling in a globalized world. Springer, Berlin. Conrad, F.G. and M.F. Schober, eds. 2008. Envisioning the survey interview of the future. Wiley, New York. Converse, J.M. 2009 reprint. Survey research in the United States: roots and emergence 1890–1960. Transaction Publishers, Edison, NJ. Couper, M.P. 2008. Designing effective web surveys. Cambridge Press, Cambridge. de Leeuw, E.D., J.J. Hox and D.A. Dillman, eds. 2008. International handbook of survey methodology. Lawrence Erlbaum, New York. Dillman, D.A., J.D. Smyth and L.M. Christian. 2009. Internet, mail and mixed-mode surveys: the tailored design method (3rd ed.). Wiley, Hoboken, NJ. Donsbach, W. and M.W. Traugott, eds. 2008. The Sage handbook of public opinion research. Sage, Los Angeles.