Carol T. Mowbray

Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: Review and recommendations☆

Abstract Participant attrition poses a significant threat to the internal and external validity of panel studies, in part because participants who successfully complete all follow-up measurements often differ in significant ways from those respondents lost to attrition. The only certain safeguard against potential biases resulting from attrition is to ensure high interview completion rates during follow-up. Unfortunately, information about reducing preventable attrition is not discussed in most research reports and a comprehensive review paper has not yet been published. The purpose of the present paper is to provide a brief overview of how attrition can threaten the validity of panel studies and to discuss eight promising methods of minimizing attrition through the use of effective retention and tracking strategies. Attempts to reduce attrition are not always met with complete success, therefore, a brief discussion of statistical techniques to assess and correct for potential attrition biases is provided. Finally, methods of calculating attrition rates are suggested along with recommendations for future research.

Consumer-operated drop-in centers: Evaluation of operations and impact

Research on self-help for consumers of psychiatric services has focused on the operation of voluntary groups and largely ignored service programs operated by consumers. This evaluation study focused on six consumer-operated drop-in centers, each established for at least two years. These centers served a combined total of 1,445 consumers and were funded as demonstration projects by the Michigan Department of Mental Health. Structured interviews of consumer-users of these centers indicated that the program was meeting its funding intents of serving people with serious mental illness and of creating an environment promoting social support and shared problem solving. Levels of satisfaction were uniformly high; there were few differences across centers. Issues that emerged for future policy and research considerations included funding constraints, enhancing accessibility (particularly for women and people needing frequent hospitalization), variable levels of support from catchment area community mental health agencies, and determining the long-term benefits of drop-in center participation.

Tracking and Follow-Up Methods for Research On Homelessness

To conduct comprehensive treatment effectiveness studies, evaluators must frequently track, locate and interview service recipients long after program participation has ended. For some populations, this is an arduous task. But, despite wide variations in retention rates, the published literature from follow-up studies rarely provides information on their tracking, locating, or interviewing techniques. The present report concerns a 12-month follow-up study of individuals who were homeless and mentally ill, which achieved a remarkably high retention rate—over 70%. Quantitative information is presented on client characteristics related to locatability and agreement to be interviewed, as well as qualitative information on successful techniques, from a focus group session with research interviewers.

Describing the homeless mentally ill: Cluster analysis results

Presented descriptive data on a group of homeless, mentally ill individuals (N = 108) served by a two-site demonstration project, funded by NIMH. Comparing results with those from other studies of this population produced some differences and some similarities. Cluster analysis techniques were applied to the data, producing a 4-group solution. Data validating the cluster solution are presented. It is suggested that the cluster results provide a more meaningful and useful method of understanding the descriptive data. Results suggest that while the population of individuals served as homeless and mentally ill is quite heterogeneous, many have well-developed functioning skills--only one cluster, making up 35.2% of the sample, fits the stereotype of the aggressive, psychotic individual with skill deficits in many areas. Further discussion is presented concerning the implications of the cluster analysis results for demonstrating contextual effects and thus better interpreting research results from other studies and assisting in future services planning.

The challenge of outcome evaluation in homeless services: Engagement as an intermediate outcome measure

Abstract Evaluating treatment outcomes for programs serving individuals who are homeless and mentally ill presents many challenges in terms of measures available and timeframes for data collection and analysis. These difficulties pose significant obstacles in terms of responsiveness to the need for short-term justification of program funding and for program modifications to improve operations. Information is presented from a two-site intervention in Michigan, providing outreach and in vivo services to individuals who are homeless and mentally ill, which measured client engagement with project services. The projects use of engagement as an intermediate outcome measure is described. Data is presented supporting the validity of the measure. Documentation is provided on the measures utility in evaluating program performance and in improving project operations. Discussion follows on interpretation of program-level and individual client differences in engagement status.

An analysis of allegations of sexual abuse in a multi-victim day-care center case

Sexual abuse in day-care centers is a phenomenon of increasing concern, both in terms of its prevalence and its serious effects on child victims. Legal intervention in this area is problematic because criminal investigations are usually beset with difficulties: Characteristics of children and of the abuse situation affect disclosure rates and credibility. Research on children as witnesses is amassing; however, so far, little systematic analysis has been conducted on multi-victim child sexual abuse cases. The present study concerns a case of sexual abuse in a day-care center with over 100 alleged victims. Criteria derived from Statement Validity Analysis (SVA) protocols were applied to aggregate record review data to assess the veritability of the abuse allegations. Individual childrens abuse disclosures were also analyzed, vis-à-vis characteristics of the children (age, gender, recency of attendance) and of the investigation (agency doing the interview, number of interviews, use of anatomically detailed dolls). The use of SVA criteria supported the veritability of allegations in this day-care abuse case. Even acknowledging the limitations of archival data collected for other purposes, the aggregate data analysis reflected consistency, logical structure, and spontaneity; however, little evidence of accommodation was found. Multivariate comparison of disclosing and nondisclosing children found an unexpected inverse relationship of disclosure with age, as well as positive associations with number of interviews and use of dolls. Alternative interpretations of these results are discussed.

Client typology based on functioning level assessments: utility for service planning and monitoring.

Cluster analysis was used to develop a six-group typology based on level of functioning data from 2,447 clients with serious mental illness served by the Michigan public mental health system. The groups are described in terms of level of functioning in 16 domains, global functioning, diagnosis, demographic characteristics and services used. A group of older, poor functioning clients with high health needs was identified. A second group of highly dysfunctional clients with few health needs was also found. Two groups of young adult “chronic” clients were found: one with extensive substance abuse problems and one with moderate substance abuse problems and extreme levels of suicidal and aggressive behavior. Two groups of relatively good functioning clients were also identified: one with some problems in many areas and a second with functional problems in a number of life areas and pronounced symptoms of depression and demoralization. The results point to the utility of cluster analysis as a mechanism to organize and identify patterns within the rich array of information provided by multidimensional level of functioning assessments. The uses of empirically derived client types in planning and monitoring mental health services are discussed.

Short versus longer term effectiveness of an outreach program for the homeless mentally Ill

Presents 12-month follow-up results from an outreach/linkage intervention with persons who are homeless and mentally ill, contrasting these with results obtained at 4 months. Both sets reflect the success of the program in placing individuals in independent housing. However, longer term data provide useful information regarding client movement patterns and increased tenure in nonhomeless living arrangements beyond the termination of specialized services. Analyses of 12-month residential outcomes identified four variables as significant predictors: recruitment source, project service duration, CMH service duration, and client age. In contrast to 4-month predictors, variables reflecting baseline client functioning were no longer significantly related to outcome, suggesting that the positive effects of the intervention may take longer to achieve with some clients. Discussion focuses on the implications of these effectiveness results for future research designs and measures as well as the utility and limitations of preexperimental approaches for evaluating innovative service models when implementation and efficacy experiences are lacking.

Project WINS: Integrating vocational services on mental health case management teams

In an effort to expand employment opportunities available to psychiatric consumers, a research demonstration project was developed to incorporate vocational services into case management teams. Following psychiatric rehabilitation principles, Project WINS employs professionals as Vocational Specialists and consumers as Peer Support Specialists to work with clients and their case managers. This report describes the characteristics of clients eligible for WINS services, the implementation of the program, and the impact of WINS on clients and on the mental health and vocational systems. It also describes the challenges and barriers this innovative service is confronting.

An analysis of treatment services provided in group homes for adults labeled mentally ill.

Although group homes for adults labeled mentally ill have been the focus of significant attention in deinstitutionalization discussions, substantive research on their operation has been lacking. The present study addresses past limitations by focusing on a well-defined sample of small group homes, with higher staffing levels and in-home treatment provided. Thirty homes were randomly selected; all home residents (N=253) were studied. Data were collected from case managers, residents, and home staff to assess residents functioning, services and treatments provided, and the environment of the home. The findings were examined in terms of adequacy and appropriateness of care provided in the home, using accepted guidelines. The results indicated that there are aspects of a homelike environment, emotional support from staff, and skill-development activities within the home. However, an institutional focus still seems prevalent, along with limited rehabilitation and socialization activities. Residents expressed high levels of satisfaction, which may reflect their dislike of the group home alternative, e.g., the state hospital. Policy implications are discussed vis-avis current efforts to move away from group homes toward supported housing initiatives.