Patrick D. Converse

Response Rates for Mixed-Mode Surveys Using Mail and E-mail/Web

This study examines response rates for mixed-mode survey implementation involving mail and e-mail/Web components. Using Dillmans Tailored Design Method, 1,500 participants were sent a survey either (a) via mail with a follow-up contact via e-mail that directed them to a Web-based questionnaire or (b) via e-mail that directed them to a Web-based questionnaire with a follow-up contact via mail. Results indicate that these mixed-mode procedures produce moderately high response rates. However, the mail survey tended to be more effective than the e-mail/Web survey, when serving either as the initial contact or as the follow-up contact. These results suggest that survey implementation involving mail followed by e-mail/Web, or even mail-only approaches, may result in larger samples than implementation involving e-mail/Web followed by mail.

A Tale of Two Tasks: Reversing the Self-Regulatory Resource Depletion Effect

This research examined the self-regulatory depletion model (e.g., M. Muraven & R. F. Baumeister, 2000). Although numerous studies support this models prediction of decrements in self-regulation across tasks, the majority of this research has relied on a single paradigm in which two tasks are performed in succession. Other work related to learned industriousness (R. Eisenberger, 1992) and adaptation-level theory (H. Helson, 1964) indicates that self-regulatory behavior may remain stable or even improve as a result of prior self-regulatory activities in situations involving additional tasks. Three studies examined these differing perspectives with 2- and 3-task designs. Results indicated that, relative to low initial self-regulatory exertion, high exertion can lead to poorer or better subsequent self-regulation. These findings are consistent with an adaptation view of self-regulation, suggesting that the depletion effect may be only part of the picture of self-regulatory behavior over time.

Comparing Personality Test Formats and Warnings: Effects on criterion-related validity and test-taker reactions

The potential for applicant response distortion on personality measures remains a major concern in high-stakes testing situations. Many approaches to understanding response distortion are too transparent (e.g., instructed faking studies) or are too subtle (e.g., correlations with social desirability measures as indices of faking). Recent research reveals more promising approaches in two methods: using forced-choice (FC) personality test items and warning against faking. The present study examined effects of these two methods on criterion-related validity and test-taker reactions. Results supported incremental validity for an FC and Likert-scale measure in warning and no-warning conditions, above and beyond cognitive ability. No clear differences emerged between the FC vs Likert measures or warning vs no-warning conditions in terms of validity. However, some evidence suggested that FC measures and warnings may produce negative test-taker reactions. We conclude with implications for implementation in selection settings.

Faking on Personality Measures: Implications for selection involving multiple predictors

The potential for faking on noncognitive measures in high stakes testing situations remains a concern for many selection researchers and practitioners. However, the majority of previous research examining the practical effects of faking on noncognitive assessments has focused on these measures in isolation, rather than the more common situation in which they are used in combination with other predictors. The present simulation examined the effects of faking on a conscientiousness measure on criterion-related validity, mean performance of those selected, and selection decision consistency when hiring decisions were based on this measure alone vs in combination with two other predictors across a range of likely selection scenarios. Overall, results indicated that including additional predictors substantially reduced – but did not eliminate – the negative effects of faking. Faking effects varied across outcomes and selection scenarios, with effects ranging from trivial to noteworthy even for multiple-predictor selection. Implications for future research and practice are discussed.

Thinking Ahead: Assuming Linear Versus Nonlinear Personality-Criterion Relationships in Personnel Selection

Recent studies suggest that the form of some personality–performance relationships may be curvilinear, meaning that traditional top-down selection is inefficient in capitalizing on underlying personality–performance relationships. This study examines how mean performance is affected by how well the selection method is aligned with the nature of personality–criterion relationships. A simulation manipulated the linearity or nonlinear inflection point of predictor–criterion relationships, and several selection approaches were implemented that varied in level of congruence with these relationships. Results indicate that incongruence can produce notable decrements in mean performance under some conditions. Some evidence also suggests that decrements can be greater when linearity is assumed but relationships are nonlinear (vs. when nonlinearity is assumed but relationships are linear), selection ratios are smaller, and a single predictor is used.

Unit and Item Nonresponses and Ancillary Information in Web- and Paper-Based Questionnaires Administered to School Counselors

This study examined unit and item nonresponse rates and ancillary information provided for Web- and paper-based versions of a questionnaire administered to school counselors. Results indicate that the unit nonresponse rate was higher for the Web-based version but that the patterns of item nonresponse found generally did not differ between the media. Implications for counselors are discussed.

Statement Desirability Ratings in Forced-Choice Personality Measure Development: Implications for Reducing Score Inflation and Providing Trait-Level Information

Recent studies have produced mixed results regarding the effectiveness of forced-choice formats in reducing faking on personality measures. The present research examined a potentially important factor influencing forced-choice measure effectiveness—namely, the manner in which statement desirability ratings used in constructing forced-choice items are obtained—with the goal of clarifying previous findings and providing guidance for those interested in developing these measures. Two studies indicated that different desirability rating instructions produce substantially different forced-choice measures that are differentially effective in reducing score inflation and retaining trait-level information under faking conditions. These studies suggest that the manner in which statement desirability ratings are obtained may be an important consideration in forced-choice measure development.

Within-Person Variability in Goal Content and Goal Frame: Amount, Patterns, Antecedents, and Consequences of Variation in Goal Orientation Over Time

Several studies have examined goal fluctuation over time, focusing on variability in goal level. This study investigated within-person variation in goal content and goal frame. Drawing from Motivated Action Theory (DeShon & Gillespie, 2005) we examined variability in goal orientation over time in terms of amount, patterns, antecedents, and consequences. Participants completed a dispositional goal orientation measure and then daily surveys assessing perceptions, behavior, and goal orientation prior to a single performance episode. Results indicated there was substantial within-person variability, there were patterns in this variability involving goal-related behaviors and dispositional goal orientation, and certain patterns predicted exam performance. These findings suggest it may be useful to focus on not only goal level but also goal content and goal frames in future goal regulation research.

Item-Level Nonresponse Rates in an Attitudinal Survey of Teachers Delivered via Mail and Web

This article presents the results of an experiment designed to compare item-level non-response rates to paper- and web-based versions of a survey questionnaire, focusing on attitudinal variables, that was administered to highly accomplished teachers. A sample of teachers reported their perceptions of professional community; half were assigned to a web-based version of the questionnaire, and the other half to a paper-based version. In both groups, the survey implementation procedures reflected Dillman’s (2007)Tailored Design Method. Item-level nonresponses were compared between groups for overall rates, and differential response rates by demographics, item position, item format (i.e., branching items and respondent-provided text), and item content. Results revealed small differences in item-level nonresponse rates, both overall and in comparisons between demographic groups. In addition, there was no evidence of differential item-level nonresponse by item position and item content. However, item-level nonresponse rates were considerably higher for fill-in-the-blank items on the web-based questionnaire. Resume Les taux de non-reponse dans un questionnaire d’attitude transmis a des professeurs par courrier et par le web Cet article presente les resultats d‘une experience concue pour comparer les taux d’absence de reponse a chaque question des versions sur papier et sur le web d‘un meme questionnaire portant sur les variables d’attitude et distribue aupres de professeurs fort accomplis. Un echantillon de professeurs ont temoigne de leurs perceptions de leur communaute professionnelle. Une moitie fut assignee a une version web du questionnaire, tandis que l‘autre moitie a repondu a une version sur papier. Pour les deux groupes, les procedures d’implantation du sondage refletaient la methode du modele personnalise de Dillman (2007). Les non-reponses aux questions furent comparees entre les groupes pour obtenir des taux globaux et les taux de reponse differentiels furent compares selon les donnees demographiques, la position de la question, le format de la question (par exemple les questions de redirection ou celles demandant au repondant de developper sa reponse) et le contenu de la question. Les resultats revelent de petites differences dans les taux de non-reponse aux questions, tant en general que dans les comparaisons entre les groupes demographiques. La position et le contenu des questions n‘a pas revele de taux differentiels d’absence de reponse aux questions. Cependant, les taux de non-reponse aux questions etaient considerablement plus eleves face aux questions a developpement du questionnaire web. Abstract Nichtantwortquoten auf Item-Level in einer Einstellungsbefragung (Mail und Web) von Lehrern Dieser Artikel prasentiert die Ergebnisse eines Experiments, mit Hilfe dessen die Nichtantwortquoten auf Item-Level bei papier- und webbasierten Fragebogenversionen verglichen werden sollte. Der Fragebogen fokussierte auf Einstellungsvariablen und wurde an gut ausgebildete Lehrer zugestellt. Ausgewahlte Lehrer teilten ihre Sichtweise auf die berufliche Gemeinschaft mit, wobei jeweils eine Halfte der webbasierten Gruppe, die andere Halfte der Papierversion zugeordnet wurde. In beiden Gruppen folgte die Fragebogenprozedur der Tailored Design Method nach Dillman (1980). Die Nichtantwortquoten beider Gruppen wurden bezuglich der Gesamtquote und verschiedenen Antwortraten nach demographischen Variablen, Item-Position, Item-Format (z.B. verzweigte Items und Text des Befragten), und Item-Inhalt verglichen. Die Ergebnisse zeigen geringe Unterschiede der Nichtantwortquoten im Gesamtsample und im Vergleich verschiedener soziodemographischer Gruppen. Es gab keine Belege fur unterschiedliche Nichtantwortquoten aufgrund der Position oder des Inhalts des Items. Letztendlich waren die Nichtantwortquoten bei offenen Frage-Items im webbasierten Fragebogen groser. Resumen La Frecuencia del nivel de la No-Respuesta de una Encuesta Actitudinal de Profesores Enviada a traves del Correo y la Web Este ensayo presenta los resultados de un diseno experimental que compara la frecuencia del nivel de la no-respuesta de las versiones en papel y de la web de un cuestionario de encuesta, focalizando en las variables actitudinales, que fueron administradas a profesores altamente consumados. Una muestra de profesores reporto sus percepciones sobre la comunidad profesional; la mitad fueron asignados a una version web del cuestionario, y la otra mitad a una version en papel. En ambos grupos, la implementacion de los procedimientos de la encuesta reflejo el Modelo de Diseno de Dillman (2007). El nivel de la no-respuesta fue comparado entre los grupos por sus frecuencias generales y por la frecuencia de las respuestas diferenciales basados en datos demograficos, posicion del articulo, formato del articulo (a saber, ramificacion de los articulos y el texto provisto en la respuesta), y contenido del articulo. Los resultados revelaron diferencias pequenas en el nivel de frecuencia de la no respuesta, en terminos generales y en la comparacion de grupos demograficos. Ademas, no hubo evidencia en el nivel diferencial de los articulos de no-respuesta por la posicion del articulo y el contenido del articulo. No obstante, la frecuencia del nivel del articulo de la no respuesta fue considerablemente mayor en los cuestionarios para completar que en los basados en la web. ZhaiYao Yo yak

An examination of within-person variance in contextual display rules and deviation from display rules

Contextual display rules refer to perceptions of the particular emotions that should be displayed in a specific set of circumstances. As such, it is important to examine within-person variance in such perceptions within the dynamic context of real-life organizations. To this end, experience sampling methodology was used to highlight within-person variance in display rules and examine event-level predictors of these contextual display rules. Thirty-nine university staff members were surveyed four times per day about their most recent interaction. Employee momentary affect (pleasantness and activation) and relationship with the interaction target (solidarity and relative power) were examined as predictors of both contextual display rule perceptions and deviation from those display rules. Pleasantness of affect related positively to the level of expression allowed by the display rule, whereas activation of affect and target solidarity related positively to deviation from display rules. Complex interactions were also found between the predictors, further highlighting the complexity of the emotional labour process.