Ingrid K. Weigold

Examination of the Equivalence of Self-report Survey-based Paper-and-pencil and Internet Data Collection Methods

Self-report survey-based data collection is increasingly carried out using the Internet, as opposed to the traditional paper-and-pencil method. However, previous research on the equivalence of these methods has yielded inconsistent findings. This may be due to methodological and statistical issues present in much of the literature, such as nonequivalent samples in different conditions due to recruitment, participant self-selection to conditions, and data collection procedures, as well as incomplete or inappropriate statistical procedures for examining equivalence. We conducted 2 studies examining the equivalence of paper-and-pencil and Internet data collection that accounted for these issues. In both studies, we used measures of personality, social desirability, and computer self-efficacy, and, in Study 2, we used personal growth initiative to assess quantitative equivalence (i.e., mean equivalence), qualitative equivalence (i.e., internal consistency and intercorrelations), and auxiliary equivalence (i.e., response rates, missing data, completion time, and comfort completing questionnaires using paper-and-pencil and the Internet). Study 1 investigated the effects of completing surveys via paper-and-pencil or the Internet in both traditional (i.e., lab) and natural (i.e., take-home) settings. Results indicated equivalence across conditions, except for auxiliary equivalence aspects of missing data and completion time. Study 2 examined mailed paper-and-pencil and Internet surveys without contact between experimenter and participants. Results indicated equivalence between conditions, except for auxiliary equivalence aspects of response rate for providing an address and completion time. Overall, the findings show that paper-and-pencil and Internet data collection methods are generally equivalent, particularly for quantitative and qualitative equivalence, with nonequivalence only for some aspects of auxiliary equivalence.

Agentic personality characteristics and coping: Their relation to trait anxiety in college students.

Anxiety and its disorders, often present before adulthood, have high personal and societal costs for men and women. This study tested a mediation model in which 3 forms of coping mediate the relation of 3 agentic personality characteristics (i.e., traits associated with the belief that people can effectively exercise control over their lives) to lower levels of anxiety within 1 subgroup of young adults (i.e., college students). The agentic personality characteristics were (a) hardiness, (b) personal growth initiative, and (c) coping self-efficacy. The forms of dispositional coping were (a) problem-focused, (b) emotion-focused, and (c) avoidant. Results suggest that agentic personality characteristics differentially relate to forms of coping and trait anxiety. In addition, coping appears to fully mediate the relations of the personality characteristics to anxiety. The results imply that agentic personality characteristics and coping are important in decreasing and/or protecting against anxiety, in part because of how they relate to forms of coping, and suggest the need for more research.

Examining tenets of personal growth initiative using the personal growth initiative scale-II.

One promising antecedent of optimal functioning is personal growth initiative (PGI), which is the active and intentional desire to grow as a person. PGI theory and its measure, the Personal Growth Initiative Scale, have consistently shown positive relations with optimal functioning and growth. Recently, the PGI theory and its measure have been revised to account for theoretical advances. Consequently, testing of the revised theory and measure is needed to assess their capacity to predict psychological functioning and growth. The current study examined 2 tenets of PGI theory in a sample of college students. Results indicated that 3 of the 4 factors of PGI were positively related to psychological well-being and negatively related to aspects of psychological distress. In addition, the same 3 factors were related to growth in a salient domain (vocational identity development) and explained variance beyond that accounted for by more stable personality traits.

Construct distinctiveness and variance composition of multi-dimensional instruments: Three short-form masculinity measures.

Focusing on a set of 3 multidimensional measures of conceptually related but different aspects of masculinity, we use factor analytic techniques to address 2 issues: (a) whether psychological constructs that are theoretically distinct but require fairly subtle discriminations by survey respondents can be accurately captured by self-report measures, and (b) how to better understand sources of variance in subscale and total scores developed from such measures. The specific measures investigated were the: (a) Male Role Norms Inventory-Short Form (MRNI-SF); (b) Conformity to Masculine Norms Inventory-46 (CMNI-46); and (c) Gender Role Conflict Scale-Short Form (GRCS-SF). Data (N = 444) were from community-dwelling and college men who responded to an online survey. EFA results demonstrated the discriminant validity of the 20 subscales comprising the 3 instruments, thus indicating that relatively subtle distinctions between norms, conformity, and conflict can be captured with self-report measures. CFA was used to compare 2 different methods of modeling a broad/general factor for each of the 3 instruments. For the CMNI-46 and MRNI-SF, a bifactor model fit the data significantly better than did a hierarchical factor model. In contrast, the hierarchical model fit better for the GRCS-SF. The discussion addresses implications of these specific findings for use of the measures in research studies, as well as broader implications for measurement development and assessment in other research domains of counseling psychology which also rely on multidimensional self-report instruments.

Examination of the Psychometric Properties of the Personal Growth Initiative Scale–II in African American College Students

Personal growth initiative (PGI), an individual’s active and intentional desire to engage in the growth process, has been an important construct in studies of physical and mental health around the world. However, there is a dearth of research examining this construct in African American samples. In addition, PGI has recently undergone a revision of both its theory and measure; the resulting Personal Growth Initiative Scale–II (PGIS-II) has been validated for use only with European American and international college student samples. The current study examined the psychometric properties of the PGIS-II in a sample of African American college students. Confirmatory factor analyses yielded results consistent with previous studies, and the PGIS-II showed evidence of convergent and discriminant validity for three of its four factors. In addition, the PGIS-II was significantly related to aspects of Black racial identity, suggesting that it is a viable construct in this population.

Equivalence of paper-and-pencil and computerized self-report surveys in older adults

Research is lacking on the equivalence of paper-and-pencil and computer-administered surveys in older adult samples. In addition, few comparability studies have used best practices for examining equivalence. The current study investigated quantitative (means), qualitative (internal consistency and intercorrelations), and auxiliary (missing items, completion time, and comfort) equivalence for completing self-report surveys in paper-and-pencil or computerized conditions in an older adult sample. One hundred and eight older adults ages 60 and over were randomly assigned to a paper-and-pencil or computer condition and completed questionnaires assessing personality, social desirability, and computer self-efficacy. Results generally showed qualitative equivalence, with some notable differences for quantitative and auxiliary equivalence. The authors examined paper-and-pencil and computer data collection in older adults.The authors investigated the equivalence of these methods for self-report surveys.The authors tested for quantitative, qualitative, and auxiliary equivalence.As expected, results supported qualitative equivalence.As expected, results were mixed for quantitative and auxiliary equivalence.

Construct validity evidence for the Male Role Norms Inventory-Short Form: A structural equation modeling approach using the bifactor model.

The construct validity of the Male Role Norms Inventory-Short Form (MRNI-SF) was assessed using a latent variable approach implemented with structural equation modeling (SEM). The MRNI-SF was specified as having a bifactor structure, and validation scales were also specified as latent variables. The latent variable approach had the advantages of separating effects of general and specific factors and controlling for some sources of measurement error. Data (N = 484) were from a diverse sample (38.8% men of color, 22.3% men of diverse sexualities) of community-dwelling and college men who responded to an online survey. The construct validity of the MRNI-SF General Traditional Masculinity Ideology factor was supported for all 4 of the proposed latent correlations with: (a) Male Role Attitudes Scale; (b) general factor of Conformity to Masculine Norms Inventory-46; (c) higher-order factor of Gender Role Conflict Scale; and (d) Personal Attributes Questionnaire-Masculinity Scale. Significant correlations with relevant other latent factors provided concurrent validity evidence for the MRNI-SF specific factors of Negativity toward Sexual Minorities, Importance of Sex, Restrictive Emotionality, and Toughness, with all 8 of the hypothesized relationships supported. However, 3 relationships concerning Dominance were not supported. (The construct validity of the remaining 2 MRNI-SF specific factors-Avoidance of Femininity and Self-Reliance through Mechanical Skills was not assessed.) Comparisons were made, and meaningful differences noted, between the latent correlations emphasized in this study and their raw variable counterparts. Results are discussed in terms of the advantages of an SEM approach and the unique characteristics of the bifactor model. (PsycINFO Database Record

Personal Growth Initiative in the Therapeutic Process: An Exploratory Study ψ:

Personal growth initiative (PGI), an individual’s active and intentional engagement in the growth process, was originally developed as a potentially useful construct in therapy. Although it has repeatedly been related to psychological well-being and distress, few studies have examined PGI in clinical samples. The current study investigated the role of PGI in a sample of 295 clients at a community-serving training clinic. Data were collected at two time points. Confirmatory factor analyses supported a second-order model with four first-order PGI factors: Readiness for Change, Planfulness, Using Resources, and Intentional Behavior. Using cross-lagged panel analysis, PGI at Time 1 was found to predict psychological distress endorsed at Time 2 after accounting for distress at Time 1.

Factor structure of the Personal Growth Initiative Scale-II: Evidence of a bifactor model.

Personal growth initiative has been shown to be an important predictor of psychological health. It is currently measured by the Personal Growth Initiative Scale-II (PGIS-II), which consists of 4 interrelated factors. Past research across various samples has consistently selected the 4-factor model as the best fit for the data compared to single-factor and second-order models. However, its fit has typically been adequate (not strong), and, to date, no research has examined alternate factor structures, such as a bifactor solution. The current study examined 4 theoretically informed potential models—single-factor, 4-factor, second-order 4 factor, and bifactor—across 3 samples drawn from different populations: 223 college students, 307 Mechanical Turk participants, and 281 clinical therapy clients. Across all 3 samples, the bifactor model was the best fit for the data, and tests of multigroup invariance indicated this model was invariant through the scalar level. Finally, analyses of the explained common variance and percentage of uncontaminated correlations indicated that the PGIS-II can be appropriately modeled unidimensionally.

Mean scores for self-report surveys completed using paper-and-pencil and computers: A meta-analytic test of equivalence

Abstract The large body of literature on the comparability of mean scores for self-report survey responses gathered using paper-and-pencil and computer data collection methodologies has yielded inconclusive results. However, no comprehensive meta-analysis has been conducted in this field, and those that are available for specific measures have typically not differentiated between studies using between-groups and within-subjects designs. Also, few individual studies, and no meta-analyses, have used correct statistical procedures to determine the equivalence of the two methodologies. Consequently, we conducted two meta-analyses assessing quantitative equivalence (i.e., mean scores), with the first consisting of 144 independent effect sizes from studies with between-groups designs and the second including 70 independent effect sizes from studies using within-subjects designs. Both meta-analyses assessing mean scores indicated equivalence across conditions, with large heterogeneity of variance in the between-groups analysis. Presence of others in both the paper-and-pencil and computer conditions accounted for a significant portion of this variance. Heterogeneity of variance was small for the within-subjects design analysis. Overall, results indicated that the mean scores for self-report surveys using paper-and-pencil and the computer are comparable, although heterogeneity differs for the study designs. Equivalence testing was demonstrated to be the recommended statistical procedure for this type of research.