Gregory B. Teague

Fidelity Criteria: Development, Measurement, and Validation

Fidelity may be defined as the extent to which delivery of an intervention adheres to the protocol or program model originally developed. Fidelity measurement has increasing significance for evaluation, treatment effectiveness research, and service administration. Yet few published studies using fidelity criteria provide details on the construction of a valid fidelity index. The purpose of this review article is to outline steps in the development, measurement, and validation of fidelity criteria, providing examples from health and education literatures. We further identify important issues in conducting each step. Finally, we raise questions about the dynamic nature of fidelity criteria, appropriate validation and statistical analysis methods, the inclusion of structure and process criteria in fidelity assessment, and the role of program theory in deciding on the balance between adaptation versus exact replication of model programs. Further attention to the use and refinement of fidelity criteria is important to evaluation practice.

Substance abuse in schizophrenia: service utilization and costs

Utilization and cost of institutional and outpatient services were prospectively measured over 1 year for three groups of schizophrenic patients: current substance abusers, past substance abusers, and those without a history of substance abuse. Current abusers had significantly greater utilization and cost of institutional (hospital and jail) services. Current abusers also had greater utilization of emergency services. There were no significant differences between the groups in utilization and cost of other services, including psychosocial rehabilitation, outpatient treatment (case management, psychotherapy, and psychiatric visits), and housing supports. The implications for developing cost-effective treatments for dually diagnosed individuals are discussed.

Treatment of substance abuse in severely mentally ill patients

Substance abuse is the most common comorbid complication of severe mental illness. Current clinical research converges on several emerging principles of treatment that address the scope, pace, intensity, and structure of dual-diagnosis programs. They include a) assertive outreach to facilitate engagement and participation in substance abuse treatment, b) close monitoring to provide structure and social reinforcement, c) integrating substance abuse and mental health interventions in the same program, d) comprehensive, broad-based services to address other problems of adjustment, e) safe and protective living environments, f) flexibility of clinicians and programs, g) stage-wise treatment to ensure the appropriate timing of interventions, h) a longitudinal perspective that is congruent with the chronicity of dual disorders, and i) optimism.

Service system integration, access to services, and housing outcomes in a program for homeless persons with severe mental illness

OBJECTIVES This study evaluated the hypothesis that greater integration and coordination between agencies within service systems is associated with greater accessibility of services and improved client housing outcomes. METHODS As part of the Access to Community Care and Effective Services and Supports program, data were obtained on baseline client characteristics, service use, and 3-month and 12-month outcomes from 1832 clients seen at 18 sites during the first year of program operation. Data on interorganizational relationships were obtained from structured interviews with key informants from relevant organizations in each community (n = 32-82 at each site). RESULTS Complete follow-up data were obtained from 1340 clients (73%). After control for baseline characteristics, service system integration was associated with superior housing outcomes at 12 months, and this relationship was mediated through greater access to housing agencies. CONCLUSIONS Service system integration is related to improved access to housing services and better housing outcomes among homeless people with mental illness.

Delirium and other organic mental disorders in a general hospital

The authors analyze 133 cases of organic mental disorders (OMDs) from a total of 771 patients who were referred for psychiatric consultation from a general hospital. The cases represent a 2-year referral period which began July 1, 1980, when DSM-III criteria were instituted. Delirium and dementia are most commonly diagnosed and features of these, particularly in the geriatric population, are described. Delirium was more frequent in patients with multiple medical problems, was an indicator of poor prognosis having the highest mortality rate, and was usually undiagnosed by the referring physician.

The TMACT: A New Tool for Measuring Fidelity to Assertive Community Treatment

BACKGROUND: Fidelity assessment is important for implementation of evidence-based practices (EBPs), including assertive community treatment (ACT). OBJECTIVES: The TMACT, an enhanced fidelity tool, was developed and pilot-tested to better assess critical ACT structures and processes. DESIGN: Ten ACT teams were administered the TMACT and the long-standing ACT fidelity measure, the Dartmouth Assertive Community Treatment Scale (DACTS), at baseline, 6, 12, and 18 months. RESULTS: Overall, fidelity scores for all 10 teams were relatively high. Six teams showed improvement, concluding with high TMACT scores at 18 months. Four teams with significantly lower total scores had experienced turnover and organizational barriers. TMACT ratings were higher in core ACT practices than in recovery practices and EBPs. TMACT scores rose steadily but were significantly lower than DACTS scores, which remained unchanged. CONCLUSIONS: The TMACT sets higher performance standards through enhanced assessment of recovery-orientation, EBPs, and teamwork and is more sensitive to change than the DACTS.

Evidence-based practices: Setting the context and responding to concerns.

After nearly 20 years of progress in general medicine, the evidence-based practice movement is becoming the central theme for mental health care reform in the first decade of 2000. Several leaders in the movement met to discuss concerns raised by six stakeholder groups: consumers, family members, practitioners, administrators, policy makers, and researchers. Recurrent themes relate to concerns regarding the limits of science, diversion of funding from valued practices, increased costs, feasibility, prior investments in other practices, and shifts in power and control. The authors recommend that all stakeholder groups be involved in further dialog and planning to ensure that practices emerge that represent the integration of the best research evidence with clinical expertise and consumer values.

Is it ACT yet? Real-world examples of evaluating the degree of implementation for assertive community treatment

Despite growing interest in assessment of program implementation, little is known about the best way to evaluate whether a particular program has implemented the intended service to a level that is minimally acceptable to a funding source, such as a state mental health authority. Such is the case for assertive community treatment (ACT), an evidence-based practice being widely disseminated. Using an exploratory, actuarial approach to defining program standards, this study applies different statistical criteria for determining whether or not a program meets ACT standards using the 28-item Dartmouth Assertive Community Treatment Scale. The sample consists of 51 ACT programs, 25 intensive case management programs, and 11 brokered case management programs which were compared to identify levels of fidelity that discriminated between programs, but were still attainable by the majority of ACT programs. A grading system based on mean total score for a reduced set of 21 items appeared to be most attainable, but still discriminated ACT programs from other forms of case management. Implications for setting and evaluating ACT program standards are discussed.

Social networks and clinical outcomes of dually diagnosed homeless persons

Longitudinal associations between social network characteristics and clinical outcomes were examined in a group of 130 homeless persons diagnosed with substance abuse and severe mental illness. Patients whose alcohol-use disorder remitted over 18 months had more frequent contact with Alcoholics Anonymous (AA) or Narcotics Anonymous (NA) members at baseline, and those whose other substance use disorder remitted had fewer substance users in their baseline social networks. Smaller social networks predicted heavier alcohol use over time which, in turn, predicted attrition of abstinence supporters and decreased multiplexity of relationships. More frequent social contact predicted improved Expanded Brief Psychiatric Rating Scale (BPRS) symptoms. Improved BPRS symptoms and greater multiplexity of social relationships predicted each other over time. More severe BPRS symptoms and attrition of abstinence supporters predicted each other, but the mechanisms of this relationship and possible associations with alcohol or drug use remain to be tested. More severe BPRS symptoms also predicted later, proportional loss of professional contacts. Addiction Severity Index (ASI) drug use showed no longitudinal associations with social network characteristics.

The MHSIP Mental Health Report Card: A Consumer Oriented Approach to Monitoring the Quality of Mental Health Plans

Health care report cards have been endorsed as a mechanism for efficiently comparing key quanttfiable aspects of performance across a range of health systems or plans. There are challenges in determining what to measure; how to gather and analyze data; and how to report, interpret, and use findings. Mental health has received little attention, and a consumer perspec tive is typically not included. The proposed MHSIP mental health report card (MMHRC) addresses these concerns. General issues for report cards are discussed, and the MMHRC is described in terms of content, data sources and quality, and analysis and reporting.