David L. Streiner

STARTING AT THE BEGINNING: AN INTRODUCTION TO COEFFICIENT ALPHA AND INTERNAL CONSISTENCY

Cronbachs α is the most widely used index of the reliability of a scale. However, its use and interpretation can be subject to a number of errors. This article discusses the historical development of a from other indexes of internal consistency (split-half reliability and Kuder-Richardson 20) and discusses four myths associated with a: (a) that it is a fixed property of the scale, (b) that it measures only the internal consistency of the scale, (c) that higher values are always preferred over lower ones, and (d) that it is restricted to the range of 0 to 1. It provides some recommendations for acceptable values of a in different situations.

The Prevalence of Anxiety and Mood Problems among Children with Autism and Asperger Syndrome

The objective of this study was to report on the prevalence and correlates of anxiety and mood problems among 9- to 14- year-old children with Asperger syndrome (AS) and high-functioning autism. Children who received a diagnosis of autism (n 40) or AS (n 19) on a diagnostic interview when they were 4 to 6 years of age were administered a battery of cognitive and behavioural measures. Families were contacted roughly 6 years later (at mean age of 12 years) and assessed for evidence of psychiatric problems including mood and anxiety disorders. Compared with a sample of 1751 community children, AS and autistic children demonstrated a greater rate of anxiety and depression problems. These problems had a significant impact on their overall adaptation. There were, however, no differences in the number of anxiety and mood problems between the AS and autistic children within this high-functioning cohort. The number of psychiatric problems was not correlated with early autistic symptoms but was predicted to a small extent by early verbal/non-verbal IQ discrepancy scores. These data indicate that high-functioning PDD children are at greater risk for mood and anxiety problems than the general population but the correlates and risk factors for these comorbid problems remain unclear.

Being Inconsistent About Consistency: When Coefficient Alpha Does and Doesn't Matter

One of the central tenets of classical test theory is that scales should have a high degree of internal consistency, as evidenced by Cronbachs α, the mean interitem correlation, and a strong first component. However, there are many instances in which this rule does not apply. Following Bollen and Lennox (1991), I differentiate between questionnaires such as anxiety or depression inventories, which are composed of items that are manifestations of an underlying hypothetical construct (i.e., where the items are called effect indicators) and those such as Scale 6 of the Minnesota Multiphasic Personality Inventory (Hathaway & McKinley, 1943) and ones used to tap quality of life or activities of daily living in which the items or subscales themselves define the construct (these items are called causal indicators). Questionnaires of the first sort, which are referred to as scales in this article, meet the criteria of classical test theory, whereas the second type, which are called indexes here, do not. I discuss the implications of this difference for how items are selected, the relationship among the items, and the statistics that should and should not be used in establishing the reliability of the scale or index.

A Randomized Trial of Dialectical Behavior Therapy Versus General Psychiatric Management for Borderline Personality Disorder

OBJECTIVE The authors sought to evaluate the clinical efficacy of dialectical behavior therapy compared with general psychiatric management, including a combination of psychodynamically informed therapy and symptom-targeted medication management derived from specific recommendations in APA guidelines for borderline personality disorder. METHOD This was a single-blind trial in which 180 patients diagnosed with borderline personality disorder who had at least two suicidal or nonsuicidal self-injurious episodes in the past 5 years were randomly assigned to receive 1 year of dialectical behavior therapy or general psychiatric management. The primary outcome measures, assessed at baseline and every 4 months over the treatment period, were frequency and severity of suicidal and nonsuicidal self-harm episodes. RESULTS Both groups showed improvement on the majority of clinical outcome measures after 1 year of treatment, including significant reductions in the frequency and severity of suicidal and nonsuicidal self-injurious episodes and significant improvements in most secondary clinical outcomes. Both groups had a reduction in general health care utilization, including emergency visits and psychiatric hospital days, as well as significant improvements in borderline personality disorder symptoms, symptom distress, depression, anger, and interpersonal functioning. No significant differences across any outcomes were found between groups. CONCLUSIONS These results suggest that individuals with borderline personality disorder benefited equally from dialectical behavior therapy and a well-specified treatment delivered by psychiatrists with expertise in the treatment of borderline personality disorder.

Figuring out factors: the use and misuse of factor analysis.

Factor analysis is a technique which is designed to reveal whether or not the pattern of responses on a number of tests can be explained by a smaller number of underlying traits or factors. Similarly, it can be used to indicate whether or not the various items on a questionnaire can be grouped into a few clusters with each cluster reflecting a different construct. As with all multivariate statistical tests, it is quite powerful and can provide much information about the instruments being used. Similarly, there are many ways it can be abused and misinterpreted. This paper will explain the basics of factor analysis and provide some guidelines relating to how the results should be reported.

Effects on quality of life with comprehensive rehabilitation after acute myocardial infarction

Abstract This investigation was designed to determine the impact of a brief period of cardiac rehabilitation, initiated within 6 weeks of acute myocardial infarction (AMI), on both disease-specific and generic health-related quality of life, exercise tolerance and return to work after AMI. With a stratified, parallel group design, 201 low-risk patients with evidence of depression or anxiety, or both, after AMI, were randomized to either an 8-week program of exercise conditioning and behavioral counseling or to conventional care. Although the differences were small, significantly greater improvement was seen in rehabilitation group patients at 8 weeks in the emotions dimension of a new disease-specific, health-related Quality of Life Questionnaire, in their state of anxiety and in exercise tolerance. All measures of health-related quality of life in both groups improved significantly over the 12-month followup period. However, the 95% confidence intervals around differences between groups at the 12-month follow-up effectively excluded sustained, clinically important benefits of rehabilitation in disease-specific (limitations, −2.70, 1.40; emotions, −4.86, 1.10, where negative values favor conventional care and positive values favor rehabilitation) and generic health-related quality of life (time trade-off, −0.062, 0.052; quality of well-being, −0.042, 0.035) or in exercise tolerance (−38.5, 52.1 kpm/min); also, return to work was similar in the 2 groups (relative risk, 0.93; confidence interval, 0.71, 1.64). It is concluded that in patients with evidence of depression or anxiety, or both, exercise conditioning and behavioral counseling after AMI was associated with an accelerated recovery in some outcome measures at 8 weeks, but by 12 months similar improvements were seen in both diseasespecific and generic health-related quality of life and in other outcome measures when compared with conventional care in this community.

Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding

Although confounding is an important problem of cohort studies, its effects can be minimised to enable valid comparison In cohort studies, who does or does not receive an intervention is determined by practice patterns, personal choice, or policy decisions. This raises the possibility that the intervention and comparison groups may differ in characteristics that affect the study outcome, a problem called selection bias. If these characteristics have independent effects on the observed outcome in each group, they will create differences in outcomes between the groups apart from those related to the interventions being assessed. This effect is known as confounding.1 In the first paper in the series we dealt with the design and use of cohort studies and how to identify selection bias.2 This paper focuses on the definition and assessment of confounders. For a characteristic to be a confounder in a particular study, it must meet two criteria.1 The first is that it must be related to the outcome in terms of prognosis or susceptibility. For example, in the study of the association between antipsychotic use and hip fracture that we considered in the first paper,2 age is known to be related to risk of hip fracture and therefore has the potential to be a confounder. The second criterion that defines a confounder is that the distribution of the characteristic is different in the groups being compared. It can differ in terms of either the mean or the degree of variation or variability in that characteristic. For example, for age to be a confounder in a cohort study, either the average age or the variation in the age in the groups being compared would have to be different. Assessing variation as well as average values is important because groups can have the same average value …

Breaking up is Hard to Do: The Heartbreak of Dichotomizing Continuous Data:

Researchers often take variables that are measured on a continuum and then break them into categories (for example, above or below some cut-point), either to place subjects into groups or as an outcome measure. In this article, we show that the rationales given for this practice are weak and that categorization results in lost information, reduced power of statistical tests, and increased probability of a Type II error. Dichotomizing a continuous variable is justified only when the distribution of that variable is highly skewed or its relation with another variable is nonlinear.

Agreement among reviewers of review articles.

OBJECTIVE To assess the consistency of an index of the scientific quality of research overviews. DESIGN Agreement was measured among nine judges, each of whom assessed the scientific quality of 36 published review articles. ITEM SELECTION: An iterative process was used to select ten criteria relative to five key tasks entailed in conducting a research overview. SAMPLE The review articles were drawn from three sampling frames: articles highly rated by criteria external to the study; meta-analyses; and a broad spectrum of medical journals. JUDGES: Three categories of judges were used: research assistants; clinicians with research training; and experts in research methodology; with three judges in each category. RESULTS The level of agreement within the three groups of judges was similar for their overall assessment of scientific quality and for six of the nine other items. With four exceptions, agreement among judges within each group and across groups, as measured by the intraclass correlation coefficient (ICC), was greater than 0.5, and 60% (24/40) of the ICCs were greater than 0.7. CONCLUSIONS It was possible to achieve reasonable to excellent agreement for all of the items in the index, including the overall assessment of scientific quality. The implications of these results for practising clinicians and the peer review system are discussed.

Clinical impact versus factor analysis for quality of life questionnaire construction

OBJECTIVE We have compared two philosophically different methods for selecting items for a disease-specific quality of life questionnaire. The impact method selects items that are most frequently perceived as important by patients whereas the psychometric method (factor analysis) selects items primarily according to their relationships with one another. PATIENTS 150 adults with symptomatic asthma and a wide range of disease severity were enrolled from asthma clinics and notices in the local media. STUDY DESIGN From a list of 152 items that are potentially troublesome to patients with asthma, the patients identified those items they had experienced in the previous year and scored the importance of each on a five-point scale. For the impact method, items that were identified most frequently and that scored the highest were included in the final instrument. For the psychometric method, factor analysis was performed after highly skewed items had been removed. Items with high factor loading were included in the final instrument. RESULTS The impact method resulted in a 32-item instrument and psychometric analysis in one with 36 items. Twenty items were common to both instruments. The psychometric approach discarded the highest impact emotional function and environmental items and included in their place lower impact items mainly associated with fatigue. CONCLUSIONS Although some items were the same for both methods, there were also some important differences. Different approaches to item reduction led to appreciably different instruments.