Patrick E. Shrout

Intraclass correlations: Uses in assessing rater reliability.

Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.

Measurement reliability and agreement in psychiatry

Psychiatric research has benefited from attention to measurement theories of reliability, and reliability/agreement statistics for psychopathology ratings and diagnoses are regularly reported in empirical reports. Nevertheless, there are still controversies regarding how reliability should be measured, and the amount of resources that should be spent on studying measurement quality in research programs. These issues are discussed in the context of recent theoretical and technical contributions to the statistical analysis of reliability. Special attention is paid to statistical studies published since Kraemers 1992 review of reliability methods in this journal.

Prevalence of Mental Illness in Immigrant and Non-Immigrant U.S. Latino Groups

OBJECTIVE Although widely reported among Latino populations, contradictory evidence exists regarding the generalizability of the immigrant paradox, i.e., that foreign nativity protects against psychiatric disorders. The authors examined whether this paradox applies to all Latino groups by comparing estimates of lifetime psychiatric disorders among immigrant Latino subjects, U.S-born Latino subjects, and non-Latino white subjects. METHOD The authors combined and examined data from the National Latino and Asian American Study and the National Comorbidity Survey Replication, two of the largest nationally representative samples of psychiatric information. RESULTS In the aggregate, risk of most psychiatric disorders was lower for Latino subjects than for non-Latino white subjects. Consistent with the immigrant paradox, U.S.-born Latino subjects reported higher rates for most psychiatric disorders than Latino immigrants. However, rates varied when data were stratified by nativity and disorder and adjusted for demographic and socioeconomic differences across groups. The immigrant paradox consistently held for Mexican subjects across mood, anxiety, and substance disorders, while it was only evident among Cuban and other Latino subjects for substance disorders. No differences were found in lifetime prevalence rates between migrant and U.S.-born Puerto Rican subjects. CONCLUSIONS Caution should be exercised in generalizing the immigrant paradox to all Latino groups and for all psychiatric disorders. Aggregating Latino subjects into a single group masks significant variability in lifetime risk of psychiatric disorders, with some subgroups, such as Puerto Rican subjects, suffering from psychiatric disorders at rates comparable to non-Latino white subjects. Our findings thus suggest that immigrants benefit from a protective context in their country of origin, possibly inoculating them against risk for substance disorders, particularly if they emigrated to the United States as adults.

A Procedure for Evaluating Sensitivity to Within-Person Change: Can Mood Measures in Diary Studies Detect Change Reliably?

The recent growth in diary and experience sampling research has increased research attention on how people change over time in natural settings. Often however, the measures in these studies were originally developed for studying between-person differences, and their sensitivity to within-person changes is usually unknown. Using a Generalizability Theory framework, the authors illustrate a procedure for developing reliable measures of change using a version of the Profile of Mood States (POMS; McNair, Lorr, & Droppleman, 1992) shortened for diary studies. Analyzing two data sets, one composed of 35 daily reports from 68 persons experiencing a stressful examination and another composed of daily reports from 164 persons over a typical 28-day period, we demonstrate that three-item measures of anxious mood, depressed mood, anger, fatigue, and vigor have appropriate reliability to detect within-person change processes.

Paper or plastic? Data equivalence in paper and electronic diaries

Concern has been raised about the lack of participant compliance in diary studies that use paper-and-pencil as opposed to electronic formats. Three studies explored the magnitude of compliance problems and their effects on data quality. Study 1 used random signals to elicit diary reports and found close matches to self-reported completion times, matches that could not plausibly have been fabricated. Studies 2 and 3 examined the psychometric and statistical equivalence of data obtained with paper versus electronic formats. With minor exceptions, both methods yielded data that were equivalent psychometrically and in patterns of findings. These results serve to at least partially mollify concern about the validity of paper diary methods.

DRINKING DURING PREGNANCY AND SPONTANEOUS ABORTION

The frequency of drinking alcohol among 616 women who aborted spontaneously (cases) was compared with that among 632 women who delivered after at least 28 weeks gestation (controls). 17.0% of cases reported drinking twice a week or more during pregnancy whereas among controls, only 8.1% of women reported drinking twice a week or more. The hypothesis that drinking during pregnancy is associated with spontaneous abortion was tested by maximum-likelihood logistic regression analysis. The adjusted-odds ratio for this association was 2.62. We estimate that more than 1/4 of pregnant women drinking twice a week or more are likely to abort, compared with about 14% among women who drink less often. Consideration of wine, beer, and spirits suggested that the minimum harmful dosage was 1 ounce of absolute alcohol. Several potentially confounding variables, including maternal age, gestation, prior spontaneous abortions, smoking, and nausea/vomiting, were controlled in the analysis. The association between drinking during pregnancy and spontaneous abortion did not vary with these factors. Even moderate alcohol consumption during pregnancy is a risk factor for, and may be a cause of, spontaneous abortion. Among the possible mechanisms, acute fetal poisoning seems the most likely, although chronic poisoning is also possible.

Risk Factors for Long-Term Homelessness: Findings From a Longitudinal Study of First-Time Homeless Single Adults

OBJECTIVES We examined risk factors for long-term homelessness among newly homeless men and women who were admitted to New York City shelters in 2001 and 2002. METHODS Interviews were conducted with 377 study participants upon entry into the shelter and at 6-month intervals for 18 months. Standardized assessments of psychiatric diagnosis, symptoms, and coping skills; social and family history; and service use were analyzed. Kaplan-Meier survival analysis and Cox regression were used to examine the association between baseline assessments and duration of homelessness. RESULTS Eighty-one percent of participants returned to community housing during the follow-up period; the median duration of homelessness was 190 days. Kaplan-Meier survival analysis showed that a shorter duration of homelessness was associated with younger age, current or recent employment, earned income, good coping skills, adequate family support, absence of a substance abuse treatment history, and absence of an arrest history. Cox regression showed that older age group P<.05) and arrest history (P<.01) were the strongest predictors of a longer duration of homelessness. CONCLUSIONS Identification of risk factors for long-term homelessness can guide efforts to reduce lengths of stay in homeless shelters and to develop new preventive interventions.

Reliability of Scales With General Structure: Point and Interval Estimation Using a Structural Equation Modeling Approach

A method for obtaining point and interval estimates of reliability for composites of measures with a general structure is discussed. The approach is based on fitting a correspondingly constrained structural equation model and generalizes earlier covariance structure analysis methods for scale reliability estimation with congeneric tests. The procedure can be used with weighted or unweighted composites, in which the weights need not be known in advance but may be estimated simultaneously. The approach allows one also to obtain an approximate standard error and confidence interval for scale reliability using the bootstrap methodology.

Receiving Support as a Mixed Blessing: Evidence for Dual Effects of Support on Psychological Outcomes

Although social support is thought to boost feelings of closeness in dyadic relationships, recent findings have suggested that support receipt can increase distress in recipients. The authors investigated these apparently contrary findings in a large daily diary study of couples over 31 days leading up to a major stressor. Results confirm that daily support receipt was associated with greater feelings of closeness and greater negative mood. These average effects, however, masked substantial heterogeneity. In particular, those recipients showing greater benefits on closeness tended to show lesser cost on negative mood, and vice versa. Self-esteem was examined as a possible moderator of support effects, but its role was evident in only a subset of recipients. These results imply that models of dyadic support processes must accord a central role to between-individual heterogeneity.

A prospective study of children with head injuries: IV specific cognitive deficits

Twenty-five children with a head injury resulting in a posttraumatic amnesia of at least one week were compared with an individually matched group of 25 children with orthopedic injuries. Both groups were studied a few weeks after the accident and the again at 4 months, 1 year and 21/4 years after the injury. The head injury group showed a persistent deficit on the WISC Performance IQ scale; the deficit on the Verbal IQ scale was more transient. An extensive battery of neuropsychological tests was employed to identify specific deficits not shown on the WISC. On the whole, these tests showed a pattern of results similar to that found with the WISC, and in most cases children without deficits on the Performance IQ scale of the WISC also did not show deficits on the battery of specific tests. However, in a few cases, tests of speed of visuo-motor or visuo-spatial functioning picked up deficits attributable to the head injury which occurred in children with normal scores on the WISC.