Steven V. Rouse

Predictive and Treatment Validity of Life Satisfaction and the Quality of Life Inventory

The clinical and positive psychology usefulness of quality of life, well-being, and life satisfaction assessments depends on their ability to predict important outcomes and to detect intervention-related change. These issues were explored in the context of a program of instrument validation for the Quality of Life Inventory (QOLI) involving 3,927 clients from various clinical settings. Clinical norms were also generated that supplement existing nationwide norms. The predictive validity of the QOLI and life satisfaction in a university counseling center was supported in terms of its ability to predict academic retention both by itself and in conjunction with cumulative grade point average 1 to 3 years in advance. The QOLI was also found to be sensitive to treatment-related change in two naturalistic clinical settings and samples. The interpretation and intervention utility of measures of quality of life, well-being, and life satisfaction are discussed with respect to clinical and positive psychology research.

A reliability analysis of Mechanical Turk data

MTurk workers completed several forms of the same self-report scale.Generally, score reliability was lower for MTurk workers than the normative sample.Survey length and payment did not affect reliability.However, more reliable scores were provided when workers verified attentiveness. Amazons Mechanical Turk (MTurk) provides researchers with access to a diverse set of people who can serve as research participants, making the process of data collection a streamlined and cost-effective one. While a small number of studies are often cited to support the use of this methodology, there remains a need for additional analyses of the quality of the research data. In the present study, MTurk-based responses for a personality scale were found to be significantly less reliable than scores previously reported for a community sample. While score reliability was not affected by the length of the survey or the payment rates, the presence of an item asking respondents to affirm that they were attentive and honest was associated with more reliable responses. Best practices for MTurk-based research and continuing research needs are addressed.

Advances in Clinical Personality Measurement: An Item Response Theory Analysis of the MMPI-2 PSY-5 Scales

Item response theory (IRT) provides valuable methods for the analysis of the psychometric properties of a psychological measure. To date, however, these methods have not been used frequently by personality assessment researchers, in part because many researchers have not been introduced to the methods and in part because most of the development of IRT has taken place in applied education assessment settings, resulting in terminology that is ability focused rather than trait focused. The purpose of this article is twofold. First, an overview of IRT is presented, highlighting the concepts of the three-parameter IRT model, item and test information, and conditional standard error of measurement. Second, the psychometric properties of the (MMPI-2) PSY-5 scales are examined to demonstrate IRTs value.

What Do the MMPI–2 Restructured Clinical Scales Reliably Measure? Answers From Multiple Research Settings

The Restructured Clinical (RC; Tellegen et al., 2003) scales were developed to improve measurement of the core constructs of the MMPI–2 (Butcher et al., 2001) Clinical scales by removing “demoralization,” hypothesized to affect these scales adversely. Using 25 samples with MMPI–2 responses from 78,159 subjects across diverse clinical settings, we found that each RC scale was highly correlated with a Supplementary, Content, or Personality Psychopathology 5 (PSY–5; Harkness, McNulty, & Ben-Porath, 1995) scale: higher, in fact, than the correlation between the RC scale and its parent scale. Furthermore, for over half the RC scales (i.e., RC1, RC3, RC7, RC8, and RCd), the correlations were strong enough to conclude that the RC scales replicate MMPI–2 scales with rich empirical foundations; the remaining RC scales were not redundant. Next, we examined reliability estimates using alpha coefficients and interitem correlations and did not reveal superior reliability for most of the RC scales over existing MMPI–2 scales.

Exploring the accuracies and inaccuracies of personality perception following Internet-mediated communication

Abstract Two studies explored personality perception following Internet-mediated chat. First, participants completed a measure of the Five Factor Model (FFM), interacted online in stranger-dyads, and then rated each other’s traits. The results showed low correspondence between self-ratings and the perceptions of online partners. Internet-veterans showed no personality perception superiority over Internet-novices, and an assumed similarity effect was observed for Extraversion. In the second study, participants completed a measure of the FFM and participated in a group chat session. Research assistants coded verbal behaviors for each chat transcript. Raters read the chat transcripts and rated the traits of the participants. Verbal behaviors were typically uncorrelated with the self-ratings of personality, but showed substantial correlations with other-ratings of personality. Self–other correlations were insignificant.

Using Reliability Generalization Methods to Explore Measurement Error: An Illustration Using the MMPI–2 PSY–5 Scales

Reliability generalization (RG) is a meta-analytic technique that allows for the systematic examination of variation in score reliability for different samples of test takers; this procedure is based on the recognition that reliability is not a stable property of a test but is sample dependent. As a demonstration of an RG analysis, I obtained 63 reliability coefficients for each of the MMPI–2 (Butcher et al., 2001) Personality Psychopathology 5 (Harkness, McNulty, & Ben-Porath, 1995) scales. The overall variability of alpha coefficients supports the argument that reliability is sample dependent and underscores the need for researchers to calculate reliability estimates based on their research samples rather than simply citing published alpha coefficients as evidence of score reliability. I observed statistically significant mean reliability differences for scores across the 5 scales, with the highest level of reliability observed for scores on the measure of Negative Emotionality and the lowest levels of reliability observed for scores on the measures of Aggression and Disconstraint. There was no evidence that the sex-composition of a sample was systematically related to score reliability, and there were no statistically significant differences in reliability between scores obtained with the English version of the test and those obtained with translated forms. However, reliability was consistently lower for scores on some scales when the data were obtained in nonclinical settings as opposed to clinical ones. Sample size was not significantly correlated with reliability estimates. RG methods have the potential for deepening the level of understanding about the role of reliability in the evaluation and use of personality tests.

The MMPI–2 Restructured Clinical (RC) Scales and Redundancy: Response to Tellegen, Ben-Porath, and Sellbom

In this article, we examine 5 criticisms of Tellegen, Ben-Porath, and Sellbom (2009/this issue) about our study demonstrating the redundant relationships of the Restructured Clinical (RC) scales with extant MMPI–2 scales. We discuss differences in univariate versus multivariate comparisons of the RC scales and our “proxy” scales using their data. We show that (a) both the RC and extant proxy scales identified in our analyses account for most of the variance in the Clinical scales; (b) the proxy scales are redundant with the RC scales; (c) the proxy scales matched the 6 RC scales in accounting for variance in the Clinical scales exactly in three cases, differed by ≤.02 in 2 cases, and reached a maximum of .11 in one case; (d) the item overlap between RC1 and HEA is not at issue but rather their correlation with Scale 1; and (e) the evidence for the construct validity of the RC scales is weak using findings on the incremental validity of RC4 as illustrative.

Predictive Validity and Sensitivity to Change in Quality of Life Assessment and Life Satisfaction: Further Studies of the Quality of Life Inventory or QOLI in Mental Health Settings

The clinical usefulness of quality of life assessments depends upon their ability to predict clinically important outcomes and to detect treatment-related change. In addition, norms for clinical — and not just nonclinical — samples are needed to demonstrate clinically significant change. These issues were explored in the context of a program of instrument validation for the QOLI or Quality of Life Inventory involving 3,927 clients from various clinical settings. Clinical norms for community mental health centers and university counseling centers were also generated that supplement existing nationwide, nonclinical norms. The predictive validity of the QOLI in a university counseling center was supported in terms of its ability to predict academic retention both by itself and in conjunction with cumulative G.P.A. one to three years in advance. The QOLI was also found to be sensitive to treatment-related change in two, naturalistic clinical settings and samples, that is, a managed care/substance abuse and a university counseling center setting. The interpretation and clinical utility of measures of quality of life and life satisfaction are discussed along with an agenda for further research.

Validation of the Attitudes Regarding Bisexuality Scale: Correlations With Ratings of a Positive Media Image of Bisexual Men and Women

An important limitation of past research concerning discrimination toward nonheterosexual groups has been the lack of differentiation between attitudes toward homosexuality versus bisexuality. The current researchers addressed this limitation by validating a scale measuring attitudes toward bisexuality, the Attitudes Regarding Bisexuality Scale (ARBS). Two studies validated the male and female subscales of the measure. Participants included 57 individuals for the first study and 38 for the second, all of whom were recruited using an online clearinghouse. Three surveys were given to participants to assess how ARBS scores correlated with character ratings of a bisexual media figure. Results showed that ARBS scores covaried with the ratings: low ARBS scores were associated with less favorable ratings, even when assessing the most positive bisexual representation recalled. Internal consistency estimates were also evaluated and determined to be strong.

Universal Worth: Construct and Scale Development

Although research has demonstrated that narcissistic, contingent, and unsubstantiated forms of self-esteem correlate with undesirable behavior patterns, other researchers have searched for prosocial forms of belief in ones worth. Universal worth is proposed as the belief that (a) one is valued by a deity; (b) ones value is not contingent on success or failure; and (c) one is not valued by a deity more or less than others are valued. The Universal Worth Scale (UWS) was developed to measure this set of beliefs. Psychometric analyses showed that UWS scores were both internally consistent and temporally stable. Although scores on this scale were correlated with measures of constructs that were expected to be associated, the pattern of correlations suggests that UWS scores measure a distinct construct. Future research is suggested to explore universal worth in relation to sociometer theory and terror management theory, and within the continued search for prosocial foundations of self-worth.