Maxine Krengel

Clinical Supervision Research From 1981 to 1993" A Methodological Critique

Michael V. Ellis University at Albany, State University of New York Nicholas Ladany Temple University Maxine Krengel Boston Veterans Affairs Medical Center and Boston University School of Medicine Deborah Schult University at Albany, State University of New York The empirical studies in clinical supervision published from 1981 through 1993 were investigated to assess scientific rigor and to test whether the quality of methodology had improved since the review by R. K. Russell, A. M. Crimmings, and R. W. Lent (1984). The 144 studies were evaluated according to 49 threats to validity (T. D. Cook & D. T. Campbell, 1979; R. K. Russell et al., 1984; B. E. Wampold, B. Davis, & R. H. Good III, 1990) and 8 statistical variables (e.g., effect size, statistical power, and Type I and Type II error rates). The data revealed a shift to realistic field studies, unchecked Type I and Type II error rates, medium effect sizes, and inattention to hypothesis validity. Recommendations for designing and conducting a feasible and well-designed supervision study are offered. It can be argued that a primary goal of research in clinical supervision is to test and improve theory and to guide the practice of supervision (Ellis, 1991b). A thorough under- standing of the strengths and weaknesses of supervision research would ostensibly expand supervision theory and provide practitioners with information on how to train ef- fective counselors who, in turn, will provide more effective therapy. Although there have been numerous calls for in- creasing the scientific rigor of research on counselor super- vision and training (e.g., Ellis, 1991b; Hansen & Warner, Michael V. Ellis and Deborah Schult, Department of Counseling Psychology, University at Albany, State University of New York; Nicholas Ladany, Department of Counseling Psychology, Temple University; Maxine Krengel, Psychology Section (1168), Boston Veterans Affairs Medical Center and Department of Psychology, Boston University School of Medicine. Earlier versions of this article were presented at the 96th Annual Convention of the American Psychological Association, Atlanta, Georgia, August 1988, and at the meeting of the North Atlantic Regional Association of Counselor Education and Supervision, Albany, New York, October 1991. Maxine Krengel completed some of this research while a doc- toral student in the Department of Counseling Psychology, Uni- versity at Albany, State University of New York. We are grateful to Micki Friedlander, Richard Haase, and Erica Robbins Ellis for their insightful comments on earlier versions of this article. We express our appreciation to Eric Adams, Mafoozal Ali, Elizabeth Bhargava, Virginia Flander, David Hahn, Gohpa Khan, Michelle Mautner, Deborah Melincoff, Michael Remshard, Greg Savage, Heidi Weiss, Donna Wilson, and Bradley Wolgast for their data coding and entry assistance. Correspondence concerning this article should be addressed to Michael V. Ellis, Department of Counseling Psychology, Educa- tion 220, University at Albany, State University of New York, 1400 Washington Avenue, Albany, New York 12222. Electronic mail may be sent via Internet to me464@cnsibm.albany.edu. 35 1971; Holloway & Hosford, 1983; Russell, Crimmings, & Lent, 1984), a comprehensive and in-depth investigation of the actual state of scientific rigor has yet to be conducted. If supervision research is going to meet the goal of informing theory and practice, then a thorough assessment of its meth- odological limitations and implications is warranted. At least 32 reviews of empirically based articles pertain- ing to clinical supervision and counselor training have ap- peared in the literature. Although these reviews have made substantial contributions to the field, many did not evaluate systematically the methodological or the scientific rigor of the examined studies (e.g., Harkness & Poertner, 1989; Holloway, 1984, 1992; Holloway & Neufeldt, 1995; Kaplan, 1983; Lambert & Arnold, 1987; Leddick & Ber- nard, 1980; Liddle & Halpin, 1978; Matarazzo, 1971, 1978; Matarazzo & Garner, 1992; Matarazzo & Patterson, 1986; Russell & Petrie, 1994; Stoltenberg, McNeill, & Crethar, 1994; Yutrzenka, 1995) or did so in a cursory fashion (i.e., Baker & Daniels, 1989; Baker, Daniels, & Greeley, 1990; Ford, 1979; Hansen, Pound, & Petro, 1976; Robins, & Grimes, 1982; Hansen & Warner, 1971; Holloway & Johnston, 1985; Holloway & Wampold, 1986; Kurtz, Mar- shall, & Banspach, 1985; Loganbill, Hardy, & Delworth, 1982; Stein & Lambert, 1995; Worthington, 1987). Only four reviewers presented details of the methodological flaws encountered in the studies reviewed (Alberts & Edelstein, 1990; Avis & Sprenkle, 1990; Holloway, 1987; Russell et al., 1984). The result of not systematically evaluating the methodological issues may have erroneously led to (a) equating (or even outweighing) the findings of excellent research with poor research (Hogarty, 1989; Kline, 1983), (b) exacerbating the theoretical ambiguity in the field (Meehl, 1990), and (c) drawing inaccurate inferences and conclusions (Cooper, 1989; Ellis, 1991a). The most recent review of research in individual clinical supervision that

Psychiatric Diagnoses and Neurobehavioral Symptom Severity among OEF/OIF VA Patients with Deployment-Related Traumatic Brain Injury: A Gender Comparison

BACKGROUND Traumatic brain injury (TBI) has substantial negative implications for the post-deployment adjustment of veterans who served in Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF); however, most research on veterans has focused on males. This study investigated gender differences in psychiatric diagnoses and neurobehavioral symptom severity among OEF/OIF veterans with deployment-related TBI. METHODS This population-based study examined psychiatric diagnoses and self-reported neurobehavioral symptom severity from administrative records for 12,605 United States OEF/OIF veterans evaluated as having deployment-related TBI. Men (n = 11,951) and women (n = 654) who were evaluated to have deployment-related TBI during a standardized comprehensive TBI evaluation in Department of Veterans Affairs facilities were compared on the presence of psychiatric diagnoses and severity of neurobehavioral symptoms. FINDINGS Posttraumatic stress disorder (PTSD) was the most common psychiatric condition for both genders, although women were less likely than men to have a PTSD diagnosis. In contrast, relative to men, women were 2 times more likely to have a depression diagnosis, 1.3 times more likely to have a non-PTSD anxiety disorder, and 1.5 times more likely to have PTSD with comorbid depression. Multivariate analyses indicated that blast exposure during deployment may account for some of these differences. Additionally, women reported significantly more severe symptoms across a range of neurobehavioral domains. CONCLUSION Although PTSD was the most common condition for both men and women, it is also critical for providers to identify and treat other conditions, especially depression and neurobehavioral symptoms, among women veterans with deployment-related TBI.

Cognitive Functioning in Treatment-Seeking Gulf War Veterans: Pyridostigmine Bromide Use and PTSD

Gulf War (GW) deployed veterans have reported health symptoms since returning from the war that suggest dysfunction of the central nervous system (CNS). These symptoms include memory and concentration difficulties, fatigue, and headaches. Leading hypotheses for the etiology of these cognitive complaints include psychological factors and/or exposures to chemicals with neurotoxic properties. In this study, cognitive functioning was compared in treatment-seeking GW-deployed veterans and a treatment-seeking non–GW-deployed veteran control group. Results indicated that GW-deployed veterans performed significantly worse than the comparison group on tests of attention, visuospatial skills, visual memory, and mood. GW-deployed veterans who reported taking pyridostigmine bromide (PB) performed worse than GW-deployed veterans without PB use on executive system tasks. Treatment-seeking GW-deployed veterans with diagnoses of posttraumatic stress disorder (PTSD) did not differ on cognitive test measures compared with GW-deployed veterans without PTSD. No interaction effect of PTSD and PB use was found.

Multisensory impairment reported by veterans with and without mild traumatic brain injury history

With the use of Veterans Health Administration and Department of Defense databases of veterans who completed a Department of Veterans Affairs comprehensive traumatic brain injury (TBI) evaluation, the objectives of this study were to (1) identify the co-occurrence of self-reported auditory, visual, and vestibular impairment, referred to as multisensory impairment (MSI), and (2) examine demographic, deployment-related, and mental health characteristics that were potentially predictive of MSI. Our sample included 13,746 veterans with either a history of deployment-related mild TBI (mTBI) (n = 9,998) or no history of TBI (n = 3,748). The percentage of MSI across the sample was 13.9%, but was 17.4% in a subsample with mTBI history that experienced both nonblast and blast injuries. The factors that were significantly predictive of reporting MSI were older age, being female, lower rank, and etiology of injury. Deployment-related mTBI history, posttraumatic stress disorder, and depression were also significantly predictive of reporting MSI, with mTBI history the most robust after adjusting for these conditions. A better comprehension of impairments incurred by deployed servicemembers is needed to fully understand the spectrum of blast and nonblast dysfunction and may allow for more targeted interventions to be developed to address these issues.

Neuropsychological Screening for Cognitive Impairment Using Computer-Assisted Tasks:

The aim of this study was to validate a computer-assisted screening battery for classifying patients into two groups, those with and without cognitive impairment. Participants were all patients referred to the neuropsychology clinics at four VA medical centers during a 1-year period. Patients meeting the study inclusionary criteria (N = 252) were administered the Neurobehavioral Evaluation System-3 (NES3) computer-assisted battery. A detailed neuropsychological examination was carried out by an experienced neuropsychologist, who diagnosed the patient as cognitively impaired or not impaired. The neuropsychologists diagnosis was the gold standard. Recursive partitioning analyses yielded several classification procedures using the NES3 data to predict the gold standard. These procedures produced a set of six NES3 tasks that provide good sensitivity and specificity in predicting diagnosis. Sensitivity and specificity for the least random classification procedure were 0.87 and 0.67, respectively. The results suggest that computer-assisted screening methods are a promising means of triaging patients.

Screening for mild traumatic brain injury in OEF-OIF deployed US military: An empirical assessment of VHA's experience

Background: VHA screens for traumatic brain injury (TBI) among patients formerly deployed to Afghanistan or Iraq, referring those who screen positive for a Comprehensive TBI Evaluation (CTBIE). Methods: To assess the programme, rates were calculated of positive screens for potential TBI in the population of patients screened in VHA between October 2007 through March 2009. Rates were derived of TBI confirmed by comprehensive evaluations from October 2008 through July 2009. Patient characteristics were obtained from Department of Defense and VHA administrative data. Results: In the study population, 21.6% screened positive for potential TBI and 54.6% of these had electronic records of a CTBIE. Of those with CTBIE records, evaluators confirmed TBI in 57.7%, yielding a best estimate that 6.8% of all those screened were confirmed to have TBI. Three quarters of all screened patients and virtually all those evaluated (whether TBI was confirmed or not) had VHA care the following year. Conclusions: VHAs TBI screening process is inclusive and has utility in referring patients with current symptoms to appropriate care. More than 90% of those evaluated received further VHA care and confirmatory evaluations were associated with significantly higher average utilization. Generalizability is limited to those who seek VHA healthcare.

Testing Self-Focused Attention Theory in Clinical Supervision: Effects of Supervisee Anxiety and Performance.

Audio- or videotaping and one-way mirrors are often used in clinical supervision. Yet, the effects of audio- or videotaping on supervisees have yielded equivocal results. Some studies suggest that audio- or videotaping increases trainee anxiety and hinders performance, whereas other studies suggest negligible effects. The authors present 2 studies in which they tested self-focused attention theory (e.g., C. S. Carver & M. F. Scheier, 1982; S. Duval & R. A. Wicklund, 1972) to explain the equivocal findings. In each study, trainees were randomly assigned to 1 of 3 awareness conditions (private or public self-awareness, or subjective awareness) and conducted initial counseling sessions. Analyses of supervisee anxiety and performance found no significant differences due to self-awareness condition in either study. The results suggest that a mirror and audio- or videotaping elicit trivial aversive effects on supervisees.

Validation of the NES2 in patients with neurologic disorders

Performance on the Neurobehavioral Evaluation System (NES) has been demonstrated to be affected by exposure to a variety of neurotoxicants. However, the relation of NES subtests to CNS function has not yet been documented in patients diagnosed with neurologic disorders known to implicate specific brain substrates. A validation study of the NES2 was carried out in patients with multiple sclerosis (MS) and Parkinsons disease (PD), disorders exhibiting neuropathology at loci (white matter in MS, basal ganglia in PD) believed to be the sites of action of several known neurotoxicants. The results indicated that performance on certain NES2 subtests was affected in expected ways in both types of patients. However, performance on many more subtests was impaired, relative to controls, in MS than in PD. The relative insensitivity to PD suggests that expansion and refinement of the battery are required if it is to serve well in detecting the effects of toxicants in subjects without frank physical symptoms. These are the goals of a new version of the instrument currently under development (NES3).

Self-Report Measures to Identify Post Traumatic Stress Disorder and/or Mild Traumatic Brain Injury and Associated Symptoms in Military Veterans of Operation Enduring Freedom (OEF)/Operation Iraqi Freedom (OIF)

Individuals serving in Iraq and Afghanistan sustain injuries associated with physical and psychological trauma. Among such injuries, mild traumatic brain injury (mTBI) and post traumatic stress disorder (PTSD) are common. Self-report measures are frequently used to identify mTBI and/or PTSD and symptoms associated with these conditions. In addition to providing information regarding mTBI and PTSD, the goal of this literature review was to identify and present information on the psychometric properties of measures used to obtain information regarding these common conditions among Veterans who have returned from Operation Enduring Freedom (OEF)/Operation Iraqi Freedom (OIF). A comprehensive review of studies in which self-report measures were used to evaluate mTBI, PTSD, and associated symptoms among OEF/OIF Veterans is presented. Findings suggest that additional work is needed to identify psychometrically sound and clinically useful self-report measures that assess mTBI and PTSD and associated symptoms among OEF/OIF Veterans.

Associations Between Traumatic Brain Injury, Suspected Psychiatric Conditions, and Unemployment in Operation Enduring Freedom/Operation Iraqi Freedom Veterans.

Objective:To examine the relations among demographic characteristics, traumatic brain injury (TBI) history, suspected psychiatric conditions, current neurobehavioral health symptoms, and employment status in Veterans evaluated for TBI in the Department of Veterans Affairs. Study Design:Retrospective cross-sectional database review of comprehensive TBI evaluations documented between October 2007 and June 2009. Participants:Operation Enduring Freedom/Operation Iraqi Freedom Veterans (n = 11 683) who completed a comprehensive TBI evaluation. Main Measures:Veterans Affairs clinicians use the comprehensive TBI evaluations to obtain information about TBI-related experiences, current neurobehavioral symptoms, and to identify suspected psychiatric conditions. Results:Approximately one-third of Veterans in this sample were unemployed, and of these, the majority were looking for work. After simultaneously adjusting for health and deployment-related variables, significant factors associated with unemployment included one or more suspected psychiatric conditions (eg, posttraumatic stress disorder, anxiety, depression), neurobehavioral symptom severity (ie, affective, cognitive, vestibular), former active duty status, injury etiology, age, lower education, and marital status. The associations of these factors with employment status varied by deployment-related TBI severity. Conclusions:Simultaneously addressing health-related, educational, and/or vocational needs may fill a critical gap for helping Veterans readjust to civilian life and achieve their academic and vocational potential.