Arne Weigold

Examination of the Equivalence of Self-report Survey-based Paper-and-pencil and Internet Data Collection Methods

Self-report survey-based data collection is increasingly carried out using the Internet, as opposed to the traditional paper-and-pencil method. However, previous research on the equivalence of these methods has yielded inconsistent findings. This may be due to methodological and statistical issues present in much of the literature, such as nonequivalent samples in different conditions due to recruitment, participant self-selection to conditions, and data collection procedures, as well as incomplete or inappropriate statistical procedures for examining equivalence. We conducted 2 studies examining the equivalence of paper-and-pencil and Internet data collection that accounted for these issues. In both studies, we used measures of personality, social desirability, and computer self-efficacy, and, in Study 2, we used personal growth initiative to assess quantitative equivalence (i.e., mean equivalence), qualitative equivalence (i.e., internal consistency and intercorrelations), and auxiliary equivalence (i.e., response rates, missing data, completion time, and comfort completing questionnaires using paper-and-pencil and the Internet). Study 1 investigated the effects of completing surveys via paper-and-pencil or the Internet in both traditional (i.e., lab) and natural (i.e., take-home) settings. Results indicated equivalence across conditions, except for auxiliary equivalence aspects of missing data and completion time. Study 2 examined mailed paper-and-pencil and Internet surveys without contact between experimenter and participants. Results indicated equivalence between conditions, except for auxiliary equivalence aspects of response rate for providing an address and completion time. Overall, the findings show that paper-and-pencil and Internet data collection methods are generally equivalent, particularly for quantitative and qualitative equivalence, with nonequivalence only for some aspects of auxiliary equivalence.

Examining tenets of personal growth initiative using the personal growth initiative scale-II.

One promising antecedent of optimal functioning is personal growth initiative (PGI), which is the active and intentional desire to grow as a person. PGI theory and its measure, the Personal Growth Initiative Scale, have consistently shown positive relations with optimal functioning and growth. Recently, the PGI theory and its measure have been revised to account for theoretical advances. Consequently, testing of the revised theory and measure is needed to assess their capacity to predict psychological functioning and growth. The current study examined 2 tenets of PGI theory in a sample of college students. Results indicated that 3 of the 4 factors of PGI were positively related to psychological well-being and negatively related to aspects of psychological distress. In addition, the same 3 factors were related to growth in a salient domain (vocational identity development) and explained variance beyond that accounted for by more stable personality traits.

Examination of the Psychometric Properties of the Personal Growth Initiative Scale–II in African American College Students

Personal growth initiative (PGI), an individual’s active and intentional desire to engage in the growth process, has been an important construct in studies of physical and mental health around the world. However, there is a dearth of research examining this construct in African American samples. In addition, PGI has recently undergone a revision of both its theory and measure; the resulting Personal Growth Initiative Scale–II (PGIS-II) has been validated for use only with European American and international college student samples. The current study examined the psychometric properties of the PGIS-II in a sample of African American college students. Confirmatory factor analyses yielded results consistent with previous studies, and the PGIS-II showed evidence of convergent and discriminant validity for three of its four factors. In addition, the PGIS-II was significantly related to aspects of Black racial identity, suggesting that it is a viable construct in this population.

Equivalence of paper-and-pencil and computerized self-report surveys in older adults

Research is lacking on the equivalence of paper-and-pencil and computer-administered surveys in older adult samples. In addition, few comparability studies have used best practices for examining equivalence. The current study investigated quantitative (means), qualitative (internal consistency and intercorrelations), and auxiliary (missing items, completion time, and comfort) equivalence for completing self-report surveys in paper-and-pencil or computerized conditions in an older adult sample. One hundred and eight older adults ages 60 and over were randomly assigned to a paper-and-pencil or computer condition and completed questionnaires assessing personality, social desirability, and computer self-efficacy. Results generally showed qualitative equivalence, with some notable differences for quantitative and auxiliary equivalence. The authors examined paper-and-pencil and computer data collection in older adults.The authors investigated the equivalence of these methods for self-report surveys.The authors tested for quantitative, qualitative, and auxiliary equivalence.As expected, results supported qualitative equivalence.As expected, results were mixed for quantitative and auxiliary equivalence.

Personal Growth Initiative in the Therapeutic Process: An Exploratory Study ψ:

Personal growth initiative (PGI), an individual’s active and intentional engagement in the growth process, was originally developed as a potentially useful construct in therapy. Although it has repeatedly been related to psychological well-being and distress, few studies have examined PGI in clinical samples. The current study investigated the role of PGI in a sample of 295 clients at a community-serving training clinic. Data were collected at two time points. Confirmatory factor analyses supported a second-order model with four first-order PGI factors: Readiness for Change, Planfulness, Using Resources, and Intentional Behavior. Using cross-lagged panel analysis, PGI at Time 1 was found to predict psychological distress endorsed at Time 2 after accounting for distress at Time 1.

Factor structure of the Personal Growth Initiative Scale-II: Evidence of a bifactor model.

Personal growth initiative has been shown to be an important predictor of psychological health. It is currently measured by the Personal Growth Initiative Scale-II (PGIS-II), which consists of 4 interrelated factors. Past research across various samples has consistently selected the 4-factor model as the best fit for the data compared to single-factor and second-order models. However, its fit has typically been adequate (not strong), and, to date, no research has examined alternate factor structures, such as a bifactor solution. The current study examined 4 theoretically informed potential models—single-factor, 4-factor, second-order 4 factor, and bifactor—across 3 samples drawn from different populations: 223 college students, 307 Mechanical Turk participants, and 281 clinical therapy clients. Across all 3 samples, the bifactor model was the best fit for the data, and tests of multigroup invariance indicated this model was invariant through the scalar level. Finally, analyses of the explained common variance and percentage of uncontaminated correlations indicated that the PGIS-II can be appropriately modeled unidimensionally.

Mean scores for self-report surveys completed using paper-and-pencil and computers: A meta-analytic test of equivalence

Abstract The large body of literature on the comparability of mean scores for self-report survey responses gathered using paper-and-pencil and computer data collection methodologies has yielded inconclusive results. However, no comprehensive meta-analysis has been conducted in this field, and those that are available for specific measures have typically not differentiated between studies using between-groups and within-subjects designs. Also, few individual studies, and no meta-analyses, have used correct statistical procedures to determine the equivalence of the two methodologies. Consequently, we conducted two meta-analyses assessing quantitative equivalence (i.e., mean scores), with the first consisting of 144 independent effect sizes from studies with between-groups designs and the second including 70 independent effect sizes from studies using within-subjects designs. Both meta-analyses assessing mean scores indicated equivalence across conditions, with large heterogeneity of variance in the between-groups analysis. Presence of others in both the paper-and-pencil and computer conditions accounted for a significant portion of this variance. Heterogeneity of variance was small for the within-subjects design analysis. Overall, results indicated that the mean scores for self-report surveys using paper-and-pencil and the computer are comparable, although heterogeneity differs for the study designs. Equivalence testing was demonstrated to be the recommended statistical procedure for this type of research.