Peter Brann

Routine outcome measurement in a child and adolescent mental health service: an evaluation of HoNOSCA

Objective: This paper evaluates a range of properties for a clinician-based instrument designed for routine use in a child and adolescent mental health service (CAMHS). Method: The Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) is a new outcome measure with great promise. Case vignettes were used to examine interrater reliability. HoNOSCA was implemented for routine outpatient use by multidisciplinary staff with a return rate of 84%. The 305 ratings obtained at assessment were analysed by age, gender and diagnosis. Asample of 145 paired ratings with a 3-month interval were examined for the measurement of change over time. Results: Interrater reliability of the total score indicates moderate reliability if absolute scores are used and good reliability if the total score is used for relative comparisons. Most scales have good to very good reliability. The scales discriminated between age and gender in the expected way. HoNOSCA correlated with clinicians’ views of change and was sensitive to change over a 3-month period. The total score seemed a proxy for severity. Conclusion: Routine outcome instruments must be explored in settings where they will be used and with realistic training. HoNOSCA appears to be of value in routine outcome measurement and although questions remain about reliability and validity, the results strongly support further investigation.

Differences in Dropout between Diagnoses in Child and Adolescent Mental Health Services

Dropout from treatment is a significant problem in child and adolescent mental health services, and findings regarding the role of possible contributing factors are inconsistent. It is argued that this inconsistency may be the result of the confounding effects of different definitions of dropout, and different dropout rates for different diagnoses. A file review of 520 new cases over a 12-month period in a large Child and Adolescent Mental Health Service in Melbourne, Australia was performed. Information was collected about the intake, parents, family, child, diagnoses and treatment. A significant relationship was found between diagnosis and dropout rate, with clients experiencing family problems or conduct disorder and ADHD being more likely to dropout, and those experiencing negative life events, anxiety disorders or those not having a diagnosis being less likely to dropout. These findings offer potential directions for services to consider specific strategies for retaining their clients. Possible reasons for these findings, methodological issues and future research directions are discussed.

Cross-National Reliability of Clinician-Rated Outcome Measures in Child and Adolescent Mental Health Services

Clinician-rated measures are in extensive use as routine outcome measures in child and adolescent mental health services. We investigated cross-national differences and inter-rater reliability of the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), the Children’s Global Assessment Scale (CGAS) and the Global Assessment of Psychosocial Disability (GAPD). Thirty clinicians from 5 nations independently rated 20 written vignettes. The national groups afterwards established national consensus ratings. There were no cross-national differences in independent scores, but there were differences in national consensus scores, which were also more severe than independent scores. The ICC for the HoNOSCA total score was 0.84, for the CGAS 0.61 and for the GAPD 0.54. These measures may usefully contribute to cross-national comparison studies.

Investigating the Strengths and Difficulties of Children from Families with a Parental Mental Illness

Abstract This study investigated whether two groups of children from families with a parent with a mental illness were different to a normative group of children (no parental illness) on the Strengths and Difficulties Questionnaire (SDQ). Of the total 833 children aged 8-12 years, there were 101 children of parents with a mental illness from the general community, 134 children of parents with a mental illness who attended an intervention program and a normative sample of 598. The intervention group scored significantly higher than the other groups on all difficulties. In addition, the general community group with a parental illness scored significantly higher than those without a parental mental illness on the Total Difficulties, Emotional Symptoms and Hyperactivity subscales. These were important findings as the community and intervention children with a parent with a mental illness scored at two and three times respectively the normative level of risk to their mental health. Combined with previous risk estimates, children with a parent who attend mental health services are between two to five times more likely than the norm to be scoring in the clinical range on the SDQ. Recommendations are made regarding child morbidity, mental health policy, programs, workforce and future research.

Overview of the national mental health benchmarking project.

Objective: This paper provides an overview of the National Mental Health Benchmarking Project (NMHBP) which explored the value of benchmarking within the four main program areas of public sector mental health (general adult, child and adolescent, older persons and forensic). The NMHBP involved a series of forums which enabled participating organizations to benchmark themselves against each other, with a view to understanding variability in performance against a range of key performance indicators (KPIs). Method: Twenty-three mental health organizations took part in the NMHBP. The forums culminated in each mental health organization documenting its performance against relevant KPIs. The processes, impacts and outcomes of the NMHBP were evaluated via a multi-methods evaluation. Results: There was considerable variability across program streams for many of the KPIs, much of which could be explained by contextual factors. Within program streams there was considerable intra-organization variability. Participants found the examination of intra-organization variability on particular indicators to be useful. Conclusions: The NMHBP has shown that benchmarking is possible. Managers and clinical leaders will need to champion benchmarking and highlight its utility in relation to quality improvement and service development if the accountability goals of the Fourth National Mental Health Plan are to be realized.

On the meaning of change in a clinician's routine measure of outcome: HoNOSCA

Objective: With the advent of routine outcomes across Australia and New Zealand, clinicians, managers, parents and children will be interested in change on these measures. This paper presents a number of approaches and the implications. Method: Health of the Nations Outcome Scales for Children and Adolescents (HoNOSCA) collected during clinical practice for 911 patients were examined for changes over time, clinical significance, treatment status, effect size, and reliable and clinically significant change. Results: Statistically significant changes in symptom severity were found related to treatment status and to changes in the number of clinically significant scales. An effect size of almost one standard deviation was noted and the proportion of patients who improved was examined. While the reliable change index was calculated, there are clinical complications with this approach. The impact of the capacity to change on specific scales illustrates a critical issue in describing outcomes. Conclusion: From a number of perspectives, change in HoNOSCA total and scale scores is valid. However, several clinical dilemmas must be faced in deciding which approach should be used. The implications of these choices may affect clinicians, patients, carers and managers in understanding change.

Factors associated with dropout and diagnosis in child and adolescent mental health services

Objective: Dropout from child and adolescent mental health services has ramifications for children, families and the services themselves. Understanding the factors that are associated with dropout for different diagnoses has the potential to assist with tailoring of services to reduce dropout. The aim of the current study was to identify such factors. Method: A file audit was conducted for all referrals to a child and adolescent mental health service over a 12 month period, yielding 520 subjects for analysis (264 male, 256 female, mean age = 12.6 years). Parent, child and service variables of interest were recorded as were diagnoses, which were categorized into 25 superordinate categories. Results: Almost 50% of subjects dropped out of treatment. Factors associated with dropout varied across diagnosis, and no factor was associated with dropout for all diagnoses. Conclusion: There are differences in the factors that were associated with dropout for different disorders. This is a useful finding in terms of understanding and preventing dropout in child and adolescent mental health settings, but more research is needed.

Benchmarking child and adolescent mental health organizations

Objective: This paper describes aspects of the child and adolescent benchmarking forums that were part of the National Mental Health Benchmarking Project (NMHBP). These forums enabled participating child and adolescent mental health organizations to benchmark themselves against each other, with a view to understanding variability in performance against a range of key performance indicators (KPIs). Method: Six child and adolescent mental health organizations took part in the NMHBP. Representatives from these organizations attended eight benchmarking forums at which they documented their performance against relevant KPIs. They also undertook two special projects designed to help them understand the variation in performance on given KPIs. Results: There was considerable inter-organization variability on many of the KPIs. Even within organizations, there was often substantial variability over time. The variability in indicator data raised many questions for participants. This challenged participants to better understand and describe their local processes, prompted them to collect additional data, and stimulated them to make organizational comparisons. These activities fed into a process of reflection about their performance. Conclusions: Benchmarking has the potential to illuminate intra- and inter-organizational performance in the child and adolescent context.

Evaluating the outcomes of adolescent day programs in an Australian child and adolescent mental health service.

Adolescent day programs (ADPs) are an increasingly used approach to treating adolescents with mental health issues. However, there is a dearth of studies empirically examining the outcomes of adolescent day programs. This study retrospectively examined the mental health functioning of 84 adolescents, pre- and post-treatment, who in addition to their ongoing outpatient treatment had participated in an ADP during a five-year period. Their functioning was compared to matched adolescents who participated only in outpatient treatment during the same time period. Statistical and clinical examinations revealed the reported outcomes following ADP treatment were at least comparable, and sometimes significantly better, when compared to the reported outcomes following outpatient treatment excluding ADP involvement. This study, while having some methodological shortcomings, provides some evidence for the efficacy of ADPs.

Brief intervention: a promising framework for child and youth mental health?

There is a discrepancy between the demand for mental health treatment amongst children, young people and their carers, and the capacity of the current service system to provide evidence based interventions. Innovative models of care are required to redress this discrepancy. One such model is the single session model, which provides a single or small number of solution focused sessions targeting one or two identified problems. Single session interventions have been trialled across a range of presenting concerns including child and youth mental health services. This paper provides a rationale for offering a brief focused intervention as part of a broader Child and Youth Mental Health Service, and introduces a model of how brief intervention fits within a broader system of care.